Bitwidth-adaptive quantization-aware training based on meta-learning

Abstract

The meta-learning-based adaptive bit width quantization recognition learning method of the present invention lightweights meta-learning into a quantization recognition learning (QAT) method and enables learning while adapting to all bit widths with lower learning computational and memory costs. Additionally, the present invention integrates few-shot-based classification learning and quantization recognition learning to perform meta-learning based on optimization or metrics, thereby enabling the application of the model to all bit widths as well as target classes that were not previously learned during training.

Description

Technology Field

[0001] The present invention relates to a deep learning lightweighting technology, and more specifically, to a method for training a deep learning model using an adaptive bitwidth quantization recognition method. Background Technology

[0002] Recent developments in the field of deep learning are providing technologies that allow resource-constrained devices to be equipped with larger networks by reducing the computational cost of neural networks. These technologies include network optimization, parameter factorization, network pruning, and quantization.

[0003] In particular, quantization can significantly reduce the size, computational load, and power consumption of a model by representing the model's weights and activations with low precision, that is, a small bit width. For example, quantizing a model from 32-bit floating-point (FP32) to 8-bit integer (Int8) using a device equipped with fast arithmetic hardware for low-precision operands can significantly reduce inference latency. Such quantization can be divided into post-training quantization and quantization-aware training, which quantizes weights and activations during the training process and simulates the effects of quantization while training.

[0004] Regarding Quantization Aware Learning (QAT), there is a problem in that it is difficult to adapt models to various bit widths for platforms with different resource constraints. To address this issue, Adaptive Bitwidth Quantization Aware Learning has emerged, enabling quantization models to adapt to bit widths of varying precision.

[0005] In addition, a large amount of labeled training data is required to train a deep learning model based on supervised learning. Therefore, in order for a deep learning model to produce results with high accuracy, it is necessary to accumulate sufficient labeled data, but depending on the problem, it may not be possible to accumulate sufficient data.

[0006] Meta-learning methods have emerged as a way to train deep learning models even with limited training data. However, these meta-learning methods do not consider the problem of quantizing and lightweighting the model at all. The problem to be solved

[0007] The present invention aims to provide a method for training a model capable of adapting to quantization of all bit width combinations while minimizing the degradation of inference accuracy even when trained with lower training computation costs and storage costs than weights and conventional quantization recognition learning methods.

[0008] Additionally, another objective of the present invention is to provide a method for training a model that can adapt to the quantization of all bit width combinations and classify a given arbitrary classification class after rapidly retraining it, even when trained with lower computational and memory costs than conventional quantization recognition learning methods. means of solving the problem

[0009] A meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect of the present invention is performed in a computing device comprising one or more processors and a memory storing program instructions executable by said processors, and comprises the steps of: obtaining soft labels by predicting all data included in a batch data set through a first artificial neural network model; sampling M tasks for which quantization bitwidths are defined for each of the weights and activations; training M task models that perform the M tasks by quantizing the weights and activations with a quantization bitwidth defined for each of the batch data set, wherein the task models are trained by reflecting a first loss for ground truth and a second loss for soft labels; and updating the parameters of the first artificial neural network model by reflecting the gradients of the parameters of the M tasks.

[0010] Specifically, the step of training a task model of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect may include a step of performing quantization according to a quantization bitwidth defined for weights and activations, a step of calculating a first loss for correct answers of a batch data set, a step of calculating a second loss for soft labels of a batch data set, and a step of calculating the gradient of the parameters of the task model reflecting the first loss and the second loss.

[0011] A meta-learning-based adaptive bit width quantization recognition learning method according to a second aspect of the present invention is also a method performed in a computing device comprising one or more processors and a memory storing program instructions executable by said processors, comprising the steps of: sampling M tasks for which quantization bit widths are defined for each of the weights and activations; determining N classes to be classified by each of the M tasks; for each of the M task models performing the M tasks, constructing a batch support set and a batch query set from a training data set and initializing the parameters of a first meta-learning model as parameters of the task model, then quantizing the weights and activations with a defined quantization bit width based on Few Shot Learning and training the task model in a Model-Agnostic Meta-Learning (MAML) manner; and updating the training parameters of the first meta-learning model by reflecting the gradient of the training parameters of the M tasks.

[0012] Specifically, the step of training a task model in the MAML manner of a meta-learning-based adaptive bitwidth quantization recognition learning method according to the second aspect may include: obtaining a sub-dataset containing only data belonging to N classification classes determined from a training data set and sampling a batch support set from the sub-dataset; initializing the training parameters of the task based on the parameters of the first meta-learning model; training the batch support set by performing quantization with a defined bitwidth for each of the weights and activations, and repeating the training of the batch support set a preset number of times; performing quantization with a predetermined bitwidth for the weights and activations of the learned parameters; sampling a batch query set from the sub-dataset in which the data does not overlap with the batch support set; calculating a third loss for the batch query set; and calculating the gradient of the parameters reflecting the third loss.

[0013] A meta-learning-based adaptive bit width quantization recognition learning method according to a third aspect of the present invention is also a method performed in a computing device comprising one or more processors and a memory for storing program instructions executable by said processors, comprising the steps of determining N classes to be classified by a task, constructing a sub-data set containing only data belonging to the N classification classes determined from a training data set and sampling a batch support set from the sub-data set, sampling a batch query set from said sub-data set in which the data does not overlap with the batch support set, sampling M tasks for which quantization bit widths are defined for each of the weights and activations, setting the parameters of the task by performing quantization with a determined bit width for the weights and activations of the parameters of a second meta-learning model for each of the M task models performing the M tasks, and then training the task models using a distance-based meta-learning method for the batch support set and the batch query set, wherein the task models are trained without using backpropagation when training the batch support set, and updating the parameters of the second meta-learning model by reflecting the gradients of the parameters of the M tasks.

[0014] Specifically, the step of training a task model of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect may include: a step of setting task parameters by performing quantization with a predetermined bitwidth for the weights and activations of the parameters of the second meta-learning model; a step of obtaining prototypes in the embedding space for each classification class by learning a batch support set without using backpropagation with a distance-based meta-learning method; a step of calculating a fourth loss for a batch query set with a distance-based meta-learning method; and a step of calculating the gradient of the task parameters reflecting the fourth loss. Effects of the invention

[0015] According to the meta-learning-based adaptive bitwidth quantization recognition learning method of the present invention, a method can be provided to train a model capable of adapting to quantization of all bitwidth combinations while minimizing the degradation of inference accuracy even when training with lower training computation and memory costs than conventional quantization recognition learning methods.

[0016] In addition, according to the meta-learning-based adaptive bitwidth quantization recognition learning method of the present invention, even if the learning is performed with lower computational and memory costs than conventional quantization recognition learning methods, it is possible to train a model that adapts to the quantization of all bitwidth combinations and simultaneously classifies a given arbitrary classification class after rapidly retraining a small amount of data. Brief explanation of the drawing

[0017] FIG. 1 conceptually illustrates a meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect of the present invention. FIG. 2 illustrates a flowchart of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect of the present invention. FIG. 3 conceptually illustrates an algorithm of a meta-learning-based adaptive bitwidth quantization recognition learning method according to the first aspect of the present invention. FIG. 4 conceptually illustrates a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention. FIG. 5 illustrates a flowchart of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention. FIG. 6 conceptually illustrates an algorithm of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention. FIG. 7 conceptually illustrates a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention. FIG. 8 illustrates a flowchart of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention. FIG. 9 conceptually illustrates an algorithm of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention. Specific details for implementing the invention

[0018] The foregoing and additional aspects are embodied in the embodiments described with reference to the attached drawings. It is understood that the components of each embodiment may be combined in various ways within the embodiment unless otherwise stated or contradictory. Each block in the block diagram may represent a physical part in some cases, but in others, it may be a logical representation of a part of the function of a single physical part or a function spanning multiple physical parts. Sometimes, the entity of a block or part thereof may be a set of program instructions. These blocks may be implemented in whole or in part by hardware, software, or a combination thereof.

[0019] Meta-learning is a method for rapidly learning new tasks by training on only a small dataset. Here, tasks can include any problem solvable by machine learning, such as supervised learning or reinforcement learning. Meta-learning can be broadly classified into two types: one that trains a base model based on optimization, and another that trains based on the distance between data points mapped to a low-dimensional space. This invention is based on meta-learning.

[0020] Trained artificial neural network models are deployed and applied across various platforms, but some platforms may have limited resources. To apply trained artificial neural network models to platforms with such limited resources, various techniques for model lightweighting have emerged, and in particular, techniques utilizing quantization are widely used. Generally, quantization is a technique that quantizes the model's weights and activations (more precisely, the output values ​​of the activation function). For example, both the model's weights and activations use 32-bit floating-point types, which are converted to 8-bit integer types for quantization. Quantization techniques include post-training quantization, which quantizes the trained model without considering quantization, and quantization-aware training, which models the effects of quantization during training. This invention utilizes the QAT method.

[0021] FIG. 1 conceptually illustrates a meta-learning-based adaptive bitwidth-adaptive quantization recognition learning method according to a first aspect of the present invention. For convenience, the meta-learning-based adaptive bitwidth-adaptive quantization recognition learning method according to the first aspect is defined as MEBQAT (Meta-learning based Bitwidth-adaptive QAT). Parameters of the first neural network model of FIG. 1 this This concerns a single epoch process learned up to. The process in Fig. 1 is repeated for a fixed number of epochs. The ground truth label is is the hard label in the form of a one-hot vector for the data currently being trained. It is the soft label predicted by the current model for the data currently being trained. is a vector of probability values ​​predicted only by the forward pass in an unquantized state, that is, at full precision.

[0022] The weights and activations to be quantized are typically 32-bit floating-point type variables. The bit width combination candidates that can quantize them can each be quantized from bit width 1 to bit width 32 (FP), allowing for many combinations. Unlike conventional techniques that require QAT training for all possible combinations, MEBQAT can be applied to all possible bit width combinations without additional training by randomly selecting and training only M (e.g., 4) combinations per epoch.

[0023] Therefore, MEBQAT randomly selects M bitwidth tasks to be performed in each epoch and sequentially trains each task. In Fig. 1, Task 1 is trained using a QAT method that quantizes during training, and predicts a value ( ) calculates. MEBQAT calculates this predicted value ( ) and the correct answer( The first loss, which is the loss between ) Calculate ) and predict value( ) and soft labels( The second loss, which is the loss between ) , KD (knowledge distillation) is calculated. Backpropagation is performed based on the first loss and the second loss, and the gradients of the parameters are calculated. After training is performed identically from Task 2 to Task M as shown in Fig. 1, the parameters of the first neural network model ( )cast Update to.

[0024] FIG. 2 illustrates a flowchart of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect of the present invention, and FIG. 3 conceptually illustrates an algorithm of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect of the present invention. A meta-learning-based adaptive bitwidth quantization recognition learning method (MEBQAT) according to a first aspect of the present invention is a method performed in a computing device comprising one or more processors and a memory that stores program instructions executable by said processors.

[0025] A computing device on which the meta-learning-based adaptive bitwidth quantization recognition learning method of the present invention is performed includes at least one processor and memory. The processor can execute program instructions that implement the meta-learning-based adaptive bitwidth quantization recognition learning method of the present invention.

[0026] The memory may be volatile memory such as random access memory, non-volatile memory or a suitable combination thereof, one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media that are accessed by a processor and can store desired information, or a suitable combination thereof.

[0027] A meta-learning-based adaptive bitwidth quantization recognition learning method (MEBQAT) according to a first aspect of the present invention is performed by a computing device, comprising the steps of obtaining soft labels, task sampling, training a task model, and updating parameters of a first artificial neural network model.

[0028] The base model in MEBQAT is referred to as the first artificial neural network model, and the parameters It is initialized to. A task set for all possible bitwidth combinations for weights for quantization and activations. Defined as. Data( ) and label( Consists of ) The training data set that becomes It is defined as a batch data set ( )Is It is sampled from. Each step of the meta-learning-based adaptive bitwidth quantization recognition learning method according to the first aspect is trained for a number of epochs set for one epoch.

[0029] The step of obtaining soft labels is to obtain soft labels by predicting all data included in the batch dataset through the first artificial neural network model. When obtaining soft labels through the first artificial neural network model, quantization is not performed on the weights and activations.

[0030] Steps to obtain soft labels It can be represented as. Index is the index of the current epoch.

[0031] As explained earlier, MEBQAT does not need to train for all possible quantization bitwidths, but instead trains by sampling only M bits.

[0032] The task sampling step is a step of sampling M tasks in which a quantization bitwidth is defined for each of the weights and activations in MEBQAT. Here, M is determined to be a positive integer that is significantly smaller than the number of possible combinations, and preferably, M can be 4.

[0033] The model training step is the stage in which M task models performing M tasks in MEBQAT are trained by quantizing weights and activations using the quantization bit width defined for each batch dataset. That is, the task index If so, the task is It can be expressed as, is the quantization bit width for the weight, and is the quantization bit width for activation.

[0034] MEBQAT also calculates the first loss for the ground truth labels and the second loss for the soft labels during the model training phase, and then trains the task model by reflecting these results.

[0035] Specifically, the step of training a task model of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a first aspect may include a step of performing quantization, a step of calculating a first loss, a step of calculating a second loss, and a step of calculating a gradient.

[0036] The step of performing quantization is a step of performing quantization according to the quantization bit width defined for the weights and activations. This It can be expressed as follows. At this time is a task It is a parameter of.

[0037] The step of calculating the first loss is the first loss for the correct labels of the task model predictions for the batch data set ( This is the step of calculating ).

[0038] The first loss in MEBQAT is It can be calculated as follows.

[0039] The step of calculating the second loss is the second loss for the soft labels of the task model predictions for the batch data set ( This is the step of calculating ). In MEBQAT, the second loss is It can be calculated as follows.

[0040] The step of calculating the slope is the first loss ( ) and second loss( This is a step of calculating the gradient of the task model parameters by reflecting ).

[0041] The gradient calculated at this time is calculated using the QAT method. In MEBQAT, the gradient is It can be calculated as follows.

[0042] The step of updating the parameters of the first artificial neural network model is to update the parameters of the first artificial neural network model by reflecting the gradients of the parameters obtained for M tasks in MEBQAT. Generally, the average of the gradients of the parameters for each task is used. MEBQAT is It can be calculated as follows. At this time, is the step size.

[0043] FIG. 4 conceptually illustrates a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention. For convenience, the meta-learning-based adaptive bitwidth quantization recognition learning method according to the second aspect is defined as MEBQAT-MAML (Model-Agnostic Meta-Learning). Parameters of the first meta-learning model of FIG. 4 this This pertains to a single epoch process of learning up to [date]. The process shown in Fig. 4 is repeated for a fixed number of epochs. MEBQAT-MAML is trained using a few-shot learning method that performs N-way K-shot classification. Therefore, MEBQAT-MAML proceeds with training by dividing the training data into a support set and a query set. Additionally, since MEBQAT-MAML is trained using the QAT method, it performs quantization of weights and activations with a fixed bit width during training. Similar to the MEBQAT described earlier, MEBQAT-MAML also randomly selects only M (e.g., 4) bit width combinations for training in each epoch. Since MEBQAT-MAML is trained so that each task classifies N classes, the N classes to be classified are also randomly selected. At this time, the types of classes to be classified may differ for each task. For example, when N=3, the classification class for Task 1 can be selected to classify {cat, goat, hog}, and the classification class for Task 2 can be selected to classify {dog, shark, lion}.

[0044] MEBQAT-MAML infers a sampled support set through a task model that performs each task to predict the value ( ) is calculated. In the example of Fig. 4, the predicted value inferred by each task model for the support set is is. MEBQAT-MAML is this predicted value ( The correct answer label of ) Loss on ( ) obtain , and backpropagate to obtain the task model for the support set ( Train ).

[0045] MEBQAT-MAML infers query sets using a task model trained on a support set to predict values ​​( ) is calculated. In the example of Fig. 3, the predicted value inferred by each task model for the query set is is. MEBQAT-MAML is this predicted value ( The correct answer label of ) Third loss regarding ( ) is calculated, and backpropagated to obtain the gradient of the task model parameters from the loss on the query set. After training is performed identically from Task 1 to Task M as shown in Fig. 3, the parameters of the first meta-learning model ( )cast Update to.

[0046] FIG. 5 illustrates a flowchart of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention, and FIG. 6 conceptually illustrates an algorithm of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention. A meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect of the present invention is also a method performed in a computing device comprising one or more processors and a memory storing program instructions executable by said processors.

[0047] A meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect includes a task sampling step, a classification class determination step, a task model training step, and a first meta-learning model parameter update step.

[0048] The base model in MEBQAT-MAML is referred to as the first meta-learning model, and the parameters It is initialized to. A task set for all possible bitwidth combinations for weights for quantization and activations. Defined as. Data( ) and label( Consists of ) The training data set that becomes It is defined as follows. Each step of the meta-learning-based adaptive bitwidth quantization recognition learning method according to the second aspect is set to one epoch, and learning proceeds for a number of epochs.

[0049] Similar to MEBQAT, MEBQAT-MAML does not need to be trained on all possible quantization bitwidths, but instead proceeds with training by sampling only M.

[0050] The task sampling step is a step of sampling M tasks for which quantization bit widths are defined for each of the weights and activations in MEBQAT-MAML. Here, M is determined to be a positive integer that is significantly smaller than the number of possible combinations, and preferably, M can be 4.

[0051] MEBQAT-MAML uses bitwidth adaptive quantization and addresses the N-way K-shot classification problem, requiring the selection of a class to be classified for each task. The class determination step in MEBQAT-MAML involves determining N classes for each of the M tasks, and the classes to be classified may differ for each task.

[0052] The step of training the task model is to train each of the M task models performing M tasks in MEBQAT-MAML based on Few Shot Learning from the training dataset, while quantizing weights and activations with a defined quantization bit width and training the task model using the Model-Agnostic Meta-Learning (MAML) method.

[0053] Specifically, the step of training a task model of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a second aspect may include a step of sampling a batch support set, a task parameter initialization step, a support set learning step, a step of performing quantization on the learned parameters of the task, a query set sampling step, a step of calculating a third loss, and a step of calculating a gradient.

[0054] The step of sampling the batch support set determines the N classification classes that the task will classify and the training data set ( N classification classes determined from ) A subset containing only data belonging to ) ) find and sub-dataset( Placement support set from ) It is the step of sampling ). The process of obtaining sub-data sets is It can be represented as follows.

[0055] The task parameter initialization step is a step of initializing the task's learning parameters based on the parameters of the first meta-learning model learned so far. The task parameter initialization step It can be represented as follows.

[0056] The support set training step is a step that trains batch support sets by performing quantization with a defined bit width for each of the weights and activations. MEBQAT-MAML trains the task model on batch support sets by repeating the support set training step a preset number of times.

[0057] The support set learning step is a step of quantizing the weights and activations of the task model for each iteration ( , S is training on the support set, u is the number of iterations), and a step of calculating the loss for inference on batch support set data of the task model ( The step of updating task model parameters through ) and backpropagation ( , α It repeats the step size.

[0058] The step of performing quantization on the learned parameters of the task is a step of performing quantization on the weights and activations of the learned parameters of the task model with a fixed bit width during the support set learning step. It can be represented as follows.

[0059] The query set sampling step is a step of sampling a batch query set from the above sub-data set in which the data does not overlap with the batch support set.

[0060] The step of calculating the third loss is the step of calculating the third loss for the batch query set. The step of calculating the third loss is the loss for the correct labels of the task model predictions for the batch query set ( This is the step of calculating ). In MEBQAT-MAML, the third loss is It can be calculated as follows.

[0061] The slope calculation step is the step of calculating the slope of the parameters of the batch task model by reflecting the third loss for the batch query set. In MEBQAT-MAML, this step It can be calculated as follows.

[0062] The MAML algorithm used in the step of training the task model may be the First-Order MAML (FOMAML) algorithm. In other words, FOMAML is an algorithm that approximates the second-order gradient using the first-order method, as calculating the second-order gradient requires high computational cost.

[0063] The step of updating the parameters of the first meta-learning model is to update the parameters of the first meta-learning model by reflecting the gradients of the parameters calculated for M tasks in MEBQAT-MAML. Generally, the average of the gradients of the parameters for each task is used. MEBQAT-MAML is It can be calculated as follows. At this time, is the step size.

[0064] FIG. 7 conceptually illustrates a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention. For convenience, the meta-learning-based adaptive bitwidth quantization recognition learning method according to the third aspect is defined as MEBQAT-PN (Prototypical Network). Parameters of the second meta-learning model of FIG. 7 this This concerns a single epoch process that is trained up to [date]. The process of Fig. 7 is repeated for a fixed number of epochs. MEBQAT-PN is trained using a few-shot learning method that performs N-way K-shot classification. Therefore, MEBQAT-PN divides the training data into a support set and a query set to proceed with training. Unlike MEBQAT-MAML, MEBQAT-PN uses the same support set and query set for each task, and each task model is trained to classify the same N classes.

[0065] Therefore, MEBQAT-PN first determines N classification classes for each epoch and samples a common support set and query set for M task models to learn. At this time, the data in the support set and the data in the query set are sampled so that they do not overlap.

[0066] Since MEBQAT-PN is trained using the QAT method, it performs quantization of weights and activations with a fixed bit width during training. Similar to the MEBQAT described earlier, MEBQAT-PN also randomly selects only M (e.g., 4) bit width combinations for training in each epoch.

[0067] Prototypes within the embedding space for each classification class by performing only forward passes on the support sets sampled through the task models performing each task ( ) is calculated. In the example of Fig. 5, the prototype set obtained by each task model for the support set It is called.

[0068] MEBQAT-PN infers query sets using a task model trained on a support set to predict values ​​( ) calculates. MEBQAT-PN calculates this predicted value ( The correct answer label of ) Third loss regarding ( ) is calculated, and backpropagated to obtain the gradient of the task model parameters from the loss on the query set. After training is performed identically from Task 1 to Task M as shown in Fig. 7, the parameters of the second meta-learning model ( )cast Update to.

[0069] FIG. 8 illustrates a flowchart of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention, and FIG. 9 conceptually illustrates an algorithm of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention. A meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect of the present invention is also a method performed in a computing device comprising one or more processors and a memory storing program instructions executable by said processors.

[0070] A meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect includes a step of determining a classification class, a step of sampling a batch support set, a step of sampling a batch query set, a step of sampling M bitwidth tasks, a step of training a task model, and a step of updating second meta-learning model parameters.

[0071] The base model in MEBQAT-PN is referred to as the second meta-learning model, and the parameters It is initialized to. A task set for all possible bitwidth combinations for weights for quantization and activations. Defined as. Data( ) and label( Consists of ) The training data set that becomes It is defined as follows. Each step of the meta-learning-based adaptive bitwidth quantization recognition learning method according to the second aspect is set to one epoch, and learning proceeds for a number of epochs.

[0072] The step of determining the classification class is the step of determining the N classes that the task will classify. MEBQAT-PN uses a bitwidth adaptive quantization method and is for the N-way K-shot classification problem, so a class to be classified must be selected for each epoch. The class to be classified may differ for each epoch.

[0073] The step of sampling the batch support set involves constructing a subset containing only data belonging to N classification classes determined from the training dataset, and sampling the batch support set from the subset. That is, the step of sampling the batch support set involves the training dataset ( N classification classes determined from ) A subset containing only data belonging to ) ) find and sub-dataset( Placement support set from ) It is the step of sampling ). The process of obtaining sub-data sets is It can be represented as follows.

[0074] The step of sampling a batch query set is to sample a batch query set from the above sub-data set in which the data does not overlap with the batch support set.

[0075] The step of sampling M bit width tasks is to sample M tasks for which quantization bit widths are defined for each of the weights and activations. That is, it is a step of sampling M tasks for which quantization bit widths are defined for each of the weights and activations in MEBQAT-PN. Here, M is determined to be a positive integer that is significantly smaller than the number of possible combinations, and preferably, M can be 4.

[0076] The step of training the task model is to set the task parameters by quantizing the weights and activations of the parameters of the second meta-learning model with a fixed bit width for each of the M task models that perform M tasks, and then train the task model using a distance-based meta-learning method on the batch support set and batch query set, while training the task model without using backpropagation when training the batch support set.

[0077] Specifically, the step of training a task model of a meta-learning-based adaptive bitwidth quantization recognition learning method according to a third aspect may include a step of setting task parameters, a step of obtaining a prototype, a step of calculating a fourth loss, and a step of calculating the gradient of task parameters.

[0078] The step of setting task parameters is a step of setting task parameters by performing quantization with a fixed bit width on the weights and activations of the parameters of the second meta-learning model. In MEBQAT-PN It can be represented as follows.

[0079] The prototype generation step involves obtaining prototypes within the embedding space for each classification class by training on batch support sets using a distance-based meta-learning method without backpropagation. MEBQAT-PN embeds data to compute class-specific prototypes within the embedding space. In this step, MEBQAT-PN trains on batch support sets using only the forward pass. In other words, it does not perform backpropagation when computing prototypes. Since the prototype is calculated as the average of the embeddings for the K data points of the corresponding class in N-way K-shot classification, it can be viewed as the average position of that class within the embedding space. In MEBQAT-PN It can be represented as follows.

[0080] The step of calculating the fourth loss is to calculate the fourth loss for a batch query set using a distance-based meta-learning method. The fourth loss is calculated using the distance between the embeddings of the query data and the class prototypes. In MEBQAT-PN, the fourth loss is It can be calculated as follows. At this time, is a dummy variable, with N classification classes ( Any class belonging to ) It means ). Therefore is N classification classes ( It means the sum for all classes belonging to ) (including classes corresponding to n).

[0081] The step of calculating the gradient of the task parameters is the step of calculating the gradient of the task parameters reflecting the fourth loss. In MEBQAT-PN, this step It can be represented as follows.

[0082] It includes a step of updating the parameters of the second meta-learning model by reflecting the gradients of the parameters of M tasks.

[0083] The step of updating the parameters of the second meta-learning model is to update the parameters of the second meta-learning model by reflecting the gradients of the parameters obtained for M tasks in MEBQAT-PN. Generally, the average of the gradients of the parameters for each task is used. MEBQAT-PN is It can be calculated as follows. At this time, is the step size.

[0084] Although the present invention has been described above with reference to embodiments with reference to the accompanying drawings, it is not limited thereto and should be interpreted to encompass various modifications that can be obviously derived from them by those skilled in the art. The claims are intended to encompass such modifications.

Claims

Claim 1 A method for meta-learning-based adaptive bitwidth quantization recognition learning, comprising: a step of obtaining a soft label by predicting all data included in a batch data set through a first artificial neural network model of full precision; a step of sampling M (M is a natural number) tasks, each having a defined quantization bitwidth for weights and activations; a step of training M task models that perform the sampled M tasks by quantizing weights and activations with a quantization bitwidth defined for the batch data set, wherein the task models are trained using a first loss reflecting ground truth and a second loss reflecting soft labels obtained from the first artificial neural network model; a step of calculating a gradient for each of the M tasks based on a loss reflecting the first loss and the second loss; and a step of updating parameters of the first artificial neural network model using the average value of the M gradients calculated for each of the M tasks. Claim 2 A meta-learning-based adaptive bitwidth quantization recognition learning method, wherein the step of training a task model comprises: a step of performing quantization according to a quantization bitwidth defined for weights and activations; a step of calculating a first loss for correct answers of a batch data set; and a step of calculating a second loss for soft labels of a batch data set. Claim 3 A method performed in a computing device comprising one or more processors and memory storing program instructions executable by said processors, comprising the steps of: sampling M (where M is a natural number) tasks for which quantization bit widths for weights and activations are defined; determining N (where N is a natural number) classes for each of the sampled M tasks to classify; for each of the M task models performing the sampled M tasks, constructing a batch support set and a batch query set from a training data set, initializing the parameters of a first meta-learning model with the parameters of the task model, quantizing weights and activations with a defined quantization bit width based on Few Shot Learning, and training the task model in a Model-Agnostic Meta-Learning (MAML) manner; and for each of the sampled M tasks, calculating the gradient of the training parameters based on the loss for the batch query set. A meta-learning-based adaptive bitwidth quantization recognition learning method comprising: a step of updating the learning parameters of a first meta-learning model using the average value of the gradients of the learning parameters calculated for each of M sampled tasks. Claim 4 In claim 3, the step of training a task model in the MAML manner comprises: obtaining a subset containing only data belonging to N classification classes determined from a training data set and sampling a batch support set from the subset; initializing training parameters of the task based on parameters of a first meta-learning model; training the batch support set by performing quantization with a defined bit width for each of the weights and activations, and repeating the training of the batch support set a preset number of times; performing quantization with a predetermined bit width for the weights and activations of the learned parameters; sampling a batch query set from the subset in which the data does not overlap with the batch support set; and calculating a third loss for the batch query set; a meta-learning-based adaptive bit width quantization recognition learning method. Claim 5 A method performed in a computing device comprising one or more processors and memory storing program instructions executable by said processors, comprising: determining N (where N is a natural number) classes to be classified by a task; constructing a sub-dataset containing only data belonging to the N classification classes determined from a training dataset and sampling a batch support set from the sub-dataset; sampling a batch query set from the sub-dataset in which the data does not overlap with the batch support set; sampling M (where M is a natural number) tasks for which quantization bit widths are defined for each of the weights and activations; setting the parameters of the task for each of the M task models performing the sampled M tasks by performing quantization with a defined bit width for the weights and activations of the parameters of a second meta-learning model, and then training the task models on the batch support set and batch query set using a distance-based meta-learning method, wherein the task models are trained without using backpropagation when training the batch support set; and for each of the M sampled tasks, calculating the gradient of the parameters based on the loss for the batch query set. A meta-learning-based adaptive bitwidth quantization recognition learning method comprising: a step of updating the parameters of a second meta-learning model using the average value of the gradients of the parameters calculated for each of M sampled tasks. Claim 6 In claim 5, the step of training a task model comprises: a step of setting the task parameters by performing quantization with a predetermined bit width for the weights and activations of the parameters of a second meta-learning model; a step of obtaining a prototype in the embedding space for each classification class by training a batch support set without using backpropagation with a distance-based meta-learning method; and a step of calculating a fourth loss for a batch query set with a distance-based meta-learning method; a meta-learning-based adaptive bit width quantization recognition learning method.

Images