Method and system for compressing and fine-tuning machine learning model

Abstract

The present disclosure relates to a method for compressing and fine-tuning a machine learning model performed by at least one processor. The method for compressing and fine-tuning a machine learning model comprises the steps of: generating a quantized model by quantizing at least some of parameters of a pre-trained model; and fine-tuning the quantized model to be suitable for a target task by freezing some of the parameters of the quantized model and updating only the remaining some of the parameters of the quantized model, by using training data associated with the target task.

Description

Method and system for compressing and fine-tuning machine learning models

[0001] The present disclosure relates to a method and system for compressing and fine-tuning a machine learning model, and more particularly, to a method and system for compressing and fine-tuning a machine learning model to simultaneously achieve quantization and parameter-efficient fine-tuning of the machine learning model while improving its performance for a target task.

[0002] With the recent advancements in artificial intelligence technology, language models are particularly being actively utilized and commercialized. Language models understand the meaning of vast amounts of text expressed by people, extract and classify information contained within the text, and even directly generate text.

[0003] Recently, pre-trained language models using self-supervised learning have attracted significant attention for their superior performance, and are being utilized in diverse fields such as natural language processing, automatic speech recognition, and computer vision. However, utilizing pre-trained language models for specific tasks typically involves fine-tuning them to improve performance. However, most pre-trained language models using self-supervised learning have massive parameters, and thus, Parameter-Efficient Fine-Tuning (PEFT) methods are being studied.

[0004] To solve the above-mentioned problem, a method, a computer-readable non-transitory recording medium recording commands, and a device (system) are provided.

[0005] The present disclosure can be implemented in various ways, including a computer-readable, non-transitory recording medium having recorded thereon a method, device (system), or instructions.

[0006] According to one embodiment of the present disclosure, a method for compressing and fine-tuning a machine learning model performed by at least one processor includes the steps of generating a quantized model by quantizing at least some of the parameters of a pre-trained model, and the step of fine-tuning the quantized model to be suitable for the target task by fixing some of the parameters of the quantized model and updating only some of the remaining parameters of the quantized model using learning data associated with the target task.

[0007] A computer-readable non-transitory recording medium having recorded thereon instructions for executing a method according to one embodiment of the present disclosure on a computer is provided.

[0008] An information processing system according to one embodiment of the present disclosure includes a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, wherein the at least one program includes instructions for generating a quantized model by quantizing at least some of the parameters of a pre-trained model, and fine-tuning the quantized model to be suitable for the target task by fixing some of the parameters of the quantized model and updating only some of the remaining parameters of the quantized model using training data associated with the target task.

[0009] According to various embodiments of the present disclosure, a fine-tuned model after quantization can achieve excellent performance on a target task despite its small model size. Consequently, accurate inference can be performed at a fast inference speed using a small memory footprint.

[0010] According to various embodiments of the present disclosure, the optimizer state size can be reduced through parameter-efficient fine-tuning, and per-task checkpoint management and transition resources can be saved.

[0011] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs (referred to as “one skilled in the art”) from the description of the claims.

[0012] FIG. 1 is a diagram illustrating an overview of a method for compressing and fine-tuning a machine learning model according to one embodiment of the present disclosure.

[0013] FIG. 2 is a block diagram showing the internal configuration of an information processing system according to one embodiment of the present disclosure.

[0014] FIG. 3 is a flowchart illustrating an example of a method for compressing and fine-tuning a machine learning model according to one embodiment of the present disclosure.

[0015] FIG. 4 is a diagram illustrating an example of a method for quantizing at least some parameters of a pre-trained model according to one embodiment of the present disclosure.

[0016] FIG. 5 is a diagram illustrating an example of the structure of a quantized model according to one embodiment of the present disclosure.

[0017] FIG. 6 is a diagram illustrating an example of a method for fine-tuning a quantized model according to one embodiment of the present disclosure.

[0018] FIG. 7 is a diagram illustrating an example of a method for quantizing at least some parameters of a pre-trained model according to another embodiment of the present disclosure.

[0019] FIG. 8 is a diagram illustrating an example of a method for fine-tuning a quantized model according to another embodiment of the present disclosure.

[0020] FIG. 9 is a diagram showing an example of comparative evaluation of the performance of a model according to one embodiment of the present disclosure.

[0021] FIG. 10 is a diagram showing an example of comparing and evaluating the size of a model, the size of an optimizer, and the degree of confusion according to one embodiment of the present disclosure.

[0022] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0023] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0024] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0025] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0026] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0027] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0028] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0029] In the present disclosure, the "system" may include, but is not limited to, at least one of a server device and a cloud device. For example, the system may be comprised of one or more server devices. As another example, the system may be comprised of one or more cloud devices. As yet another example, the system may be configured and operated by a combination of a server device and a cloud device.

[0030] FIG. 1 is a diagram illustrating an overview of a compression and fine-tuning method for a machine learning model (100) according to one embodiment of the present disclosure. The machine learning model (100) may be a pre-trained machine learning model (100) including a large number of parameters. In one embodiment, the machine learning model (100) may be a zero-shot or few-shot model trained through an unsupervised learning or self-supervised learning method using unlabeled learning data or minimally labeled learning data.

[0031] In one embodiment, the size of a pre-trained machine learning model (100) can be reduced by quantizing it. For example, the size of the weights can be reduced by quantizing at least some of the weights of the machine learning model (100). Referring to the weight size graph (150), the quantized model (110) (B) can reduce the memory requirements for storing the model and improve the inference speed due to the reduced model size (weight size) compared to the pre-trained machine learning model (100) (A). However, referring to the performance score graph (140) for the target task, the performance score of the quantized model (110) (B) can be lower than that of the pre-trained machine learning model (100) (A) due to the reduced precision of the weights.

[0032] In one embodiment, in order to utilize the machine learning model (100) for a target task, the machine learning model (100) can be fine-tuned using training data associated with the target task. However, if all parameters of a pre-trained machine learning model (100) are fine-tuned, excessive resources such as time and cost may be consumed for fine-tuning. In addition, managing checkpoints of the fine-tuned model for each target task and switching target tasks also require a lot of resources. Therefore, in one embodiment, instead of fine-tuning all parameters of the machine learning model, parameter-efficient fine-tuning can be performed by fixing some of the parameters of the machine learning model (100) and updating only some of the remaining parameters.

[0033] Referring to the performance score graph (140) for the target task, the fine-tuned model (120) (C) is a model adapted to suit the target task and may perform better on the target task than the pre-trained machine learning model (100) (A). However, referring to the weight size graph (150), the model size of the fine-tuned model (120) (C) for which only fine tuning was performed is the same as that of the pre-trained machine learning model (100) (A), so a large memory space may be required to store the model and the inference speed may also be slow.

[0034] By performing both compression and fine-tuning of the model, it is possible to simultaneously achieve the advantages of both the quantized model (110) and the fine-tuned model (120) when generating a fine-tuned and quantized model (130), but this is a very challenging task. As a method for generating a fine-tuned and quantized model (130), there are two routes: a first route (A->C->D) that fine-tunes and then quantizes, and a second route (A->B->C) that fine-tunes and then quantizes.

[0035] When a fine-tuned and quantized model (130) is generated by the first route (A->C->D) that quantizes after fine-tuning, fine-tuning is performed on a machine learning model (100) with a large model size, so there is a problem that the optimizer state size is relatively large and the performance score of the model drops during the process of quantizing after fine-tuning (see C->D of the performance score graph (140) for the target task).

[0036] Therefore, the present disclosure proposes a method for generating a fine-tuned and quantized model (130) through a second route (A->B->C) that performs fine-tuning after quantization. The fine-tuned and quantized model (130) generated through the method of the present disclosure can exhibit excellent performance for a target task despite its small model size. Accordingly, accurate inference can be performed at a fast inference speed using a small memory space. Furthermore, according to various embodiments of the present disclosure, the optimizer state size can be reduced through parameter-efficient fine-tuning, and checkpoint management and transition resources for each target task can be saved.

[0037] FIG. 2 is a block diagram illustrating the internal configuration of an information processing system (200) according to one embodiment of the present disclosure. The information processing system (200) may include a memory (210), a processor (220), a communication module (230), and an input / output interface (240). The information processing system (200) may be configured to communicate information and / or data via a network using the communication module (230).

[0038] The memory (210) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (210) may include a non-permanent mass storage device such as a read-only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a non-permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the information processing system (200) as a separate permanent storage device distinct from the memory. In addition, the memory (210) may store an operating system and at least one program code (e.g., code for compressing and fine-tuning a machine learning model installed and run in the information processing system (200).

[0039] These software components may be loaded from a computer-readable recording medium separate from the memory (210). This separate computer-readable recording medium may include a recording medium directly connectable to the information processing system (200), for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (210) through a communication module (230) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (210) based on a computer program (e.g., a program for compressing and fine-tuning a machine learning model, etc.) that is installed by files provided by developers or a file distribution system that distributes installation files of applications through the communication module (230).

[0040] The processor (220) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to a user terminal (not shown) or another external system via the memory (210) or the communication module (230). For example, the processor (220) may generate a quantized model by quantizing at least some of the parameters of a pre-trained model, and may fine-tune the quantized model to be suitable for the target task by fixing some of the parameters of the quantized model and updating only some of the remaining parameters of the quantized model using training data associated with the target task.

[0041] The communication module (230) may provide a configuration or function for the information processing system (200) to communicate with external devices, etc., through a network, and may provide a configuration or function for the information processing system (200) to communicate with an external system (e.g., a separate cloud system, etc.). For example, control signals, commands, data, etc. provided under the control of the processor (220) of the information processing system (200) may be transmitted to the external device and / or the external system through the communication module (230) and the network through the communication module of the external device and / or the external system.

[0042] In addition, the input / output interface (240) of the information processing system (200) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (200) or that the information processing system (200) may include. In FIG. 2, the input / output interface (240) is illustrated as an element configured separately from the processor (220), but is not limited thereto, and the input / output interface (240) may be configured to be included in the processor (220). The information processing system (200) may include more components than those illustrated in FIG. 2. However, there is no need to explicitly illustrate most of the conventional technology components.

[0043] FIG. 3 is a flowchart illustrating an example of a method (300) for compressing and fine-tuning a machine learning model according to one embodiment of the present disclosure. According to one embodiment, the method (300) may generate a quantized model by having a processor (e.g., at least one processor of an information processing system) quantize at least some of the parameters of a pre-trained model (S310). For example, the processor may convert at least some of the parameters of a first precision included in the pre-trained model into a combination of parameters of a second precision lower than the first precision and a scaling factor. In one embodiment, at least some of the parameters of the first precision that are converted into a combination of parameters of the second precision and a scaling factor may be linear transformation weights. Additionally, in one embodiment, activations may not be quantized to the second precision.

[0044] As a specific example of converting at least some of the parameters of the first precision into a combination of parameters of the second precision and a scaling factor, the processor may convert each of at least some of the parameters of the first precision included in the pre-trained model into a combination of a predetermined number of binary values ​​and a scaling factor. As another specific example, the processor may convert each of at least some of the parameters of the first precision included in the pre-trained model into a combination of integers and a scaling factor. Here, the integers to be converted may be integers within a range expressible by a predetermined number of bits.

[0045] In one embodiment, in a quantized model, multiple parameters among the second precision parameters may share the same scaling factor. For example, multiple weights belonging to the same row of the same weight matrix among the second precision parameters may share the same scaling factor. A specific example of a method for generating a quantized model is described in more detail below with reference to FIGS. 4 and 8 .

[0046] Then, the processor can fine-tune the quantized model to be suitable for the target task by fixing some of the parameters of the quantized model and updating only some of the parameters of the quantized model using training data associated with the target task (S320). For example, the processor can fine-tune the quantized model by fixing the parameters of the second precision among the quantized model and updating only the scaling factor. As a specific example, the processor can fine-tune the quantized model by fixing the parameters of the second precision among the linear transformation weights converted into a combination of the parameters of the second precision and the scaling factor and updating only the scaling factor. Here, other parameters except the scaling factor may be fixed and not updated. may be fixed and not updated in the fine-tuning step.

[0047] Below, two representative examples will be used to illustrate specific examples of the compression and fine-tuning methods of the machine learning model of the present disclosure. While a Transformer-based language model (e.g., an OPT model, a LLaMA model, etc.) is used as an example of a pre-trained machine learning model, this is merely an example and is not limiting; the methods of the present disclosure can be applied to any machine learning model.

[0048] FIG. 4 is a diagram illustrating an example of a method for quantizing at least some parameters of a pre-trained model according to an embodiment of the present disclosure. According to an embodiment, an information processing system may generate a quantized model by converting at least some of the parameters of a first precision included in the pre-trained model into a combination of parameters of a second precision lower than the first precision and a scaling factor. For example, the information processing system may convert linear transformation weights of a linear layer included in the pre-trained model into a combination of a predetermined number of binary values ​​and a scaling factor. Here, a plurality of binary values ​​may share the same scaling factor. For example, a linear transformation weight vector of a first precision can be quantized through the following mathematical formula 1.

[0049]

[0050] Here, is the number of quantization bits, Is A scaling factor shared by the binary weights of the dog, represents a binary vector. Here, the number of quantization bits ( ), the number of binary weights that share the same scaling factor ( ) is a hyperparameter that can be determined in advance, and There are various methods (e.g., least squares method ( =1), heuristic methods such as greedy algorithms and iterative fine-tuning methods ( >1) etc.) can be initialized by.

[0051] Similarly, the linear transformation weight matrix of first precision can be quantized through the following mathematical formula 2.

[0052]

[0053] Here, , And, represents a function of a vector that outputs a zero matrix excluding the vector elements on the diagonal. In one embodiment, the same binary weight matrix ( ) can share the same scaling factor.

[0054] Figure 4 shows the various quantization bit numbers ( ), an example of quantizing the linear transformation weights (400) of the first precision of the pre-trained model is illustrated. In the illustrated examples, the 1-bit quantized weights (410), 2-bit quantized weights (420), or 3-bit quantized weights (430) are each composed of the sum of the products of one, two, or three scaling factors and binary values ​​(-1 or 1). In addition, in the illustrated examples, the same binary weight matrix ( ) share the same scaling factor. In general, the number of quantization bits ( ) increases, the compression ratio and the mean squared error (MSE) of the weights (400) of the first precision and the quantized weights (410, 420, 430) may decrease.

[0055] If the linear transformation weights are quantized through Equation 2, the linear transformation ( ) can be converted into the following mathematical formula 3.

[0056]

[0057]

[0058]

[0059] Here, And, am.

[0060] In most machine learning models, the linear transformation weights of the linear layer occupy a large portion of the memory requirements, so quantizing the linear transformation weights can significantly compress the model size (the size of the weights). In addition, the input of the linear transformation ( ) is not quantized, the inference speed can be improved by eliminating most of the complex operations due to the binary values ​​of the binary weight matrix. In one embodiment, quantization may not be performed on the activations for quantization quality.

[0061] FIG. 5 is a diagram showing an example of the structure of a quantized model (500) according to one embodiment of the present disclosure. As an example of the structure of the quantized model (500), a quantized transformer structure is shown. In the illustrated example, a quantized linear layer (510) in the quantized transformer structure is a linear transformation weight included in a linear layer of an existing transformer, which is a predetermined number of scaling factors ( )(512) and binary weights( )(514) can be a quantized layer by being converted into a combination of.

[0062] The information processing system can fine-tune the quantized model (500) by fixing some of the parameters of the quantized model (500) and updating only the remaining some. In the example of Fig. 5, the parameters are classified into parameters that are fixed in the fine-tuning step (gray) and learnable parameters (red). For example, the information processing system can fine-tune the quantized model (500) by fixing the remaining parameters (e.g., binary weights (514), biases (516), embedding layers (520), etc.) except for the scaling factor (512) of the linear layer (510) of the quantized model (500), and updating only the scaling factor (512). This will be described in more detail below with reference to Fig. 6.

[0063] FIG. 6 is a diagram illustrating an example of a method for fine-tuning a quantized model according to one embodiment of the present disclosure. As described above with reference to FIGS. 4 and 5 , according to one embodiment, the information processing system can generate a quantized model by converting at least some of the weights (600) of a pre-trained model into a combination of a predetermined number of scaling factors (610) and binary weights (620).

[0064] The information processing system can fine-tune the quantized model by fixing the remaining parameters (e.g., binary weights (620), bias, embedding layer, etc.) except for the scaling factor (610) of the linear layer of the quantized model and updating only the scaling factor (610). For example, by using training data (630) associated with the target task (640), the quantized model can be fine-tuned to fit the target task (640) by updating only the scaling factor (610) of the quantized model. In the quantized model, the weights of the linear layer account for a large proportion, but in the case of the scaling factor (610), since multiple binary weights (620) share the same scaling factor (610), the proportion is relatively small. Therefore, the size of the learning target parameter can be significantly reduced through a fine-tuning method in which the binary weights (620) of the linear layer, which account for a large proportion in the quantized model, are fixed and only the scaling factor (610) is updated, thereby reducing the resources required for fine-tuning.

[0065] In addition, when the quantized model is fine-tuned for various target tasks (640_1, 640_2, 640_3) and utilized for various target tasks (640_1, 640_2, 640_3), the remaining parameters (e.g., binary weights (620)) except for the scaling factor (610) are fixed and shared, so only the scaling factor (610) for each target task (640_1, 640_2, 640_3) needs to be managed separately, thereby increasing management convenience.

[0066] FIG. 7 is a diagram illustrating an example of a method for quantizing at least some parameters of a pre-trained model according to another embodiment of the present disclosure. According to one embodiment, the information processing system may convert at least some of the parameters of the pre-trained model into a combination of integers and scaling factors that can be expressed in a predetermined number of bits. For example, the information processing system may convert linear transformation weights (700) of the pre-trained model into a combination of integer weights (714) and scaling factors (712). Linear transformation weights of the pre-trained model (700) is converted to a combination of integer weights (714) and scaling factors (712), resulting in a quantized weight can be expressed as in mathematical formula 4 below.

[0067]

[0068] Here, is the scaling factor, is an integer weight, is the number of quantization bits, is a rounding function, is the range It represents a clamping function. The number of quantization bits ( ) is a hyperparameter that can be determined in advance, and the scaling factor ( )(712) and integer weights( )(714) can be initialized by various methods (e.g., least squares method, etc.).

[0069] As mentioned above, linear transformation weights account for a significant portion of memory requirements. Therefore, quantizing linear transformation weights into low-bit integer weights and scaling factors can significantly compress the model size (the size of the weights). Furthermore, quantized weights increase the number of weights loaded into registers per global memory access, potentially increasing the inference speed of quantized models.

[0070] The information processing system can fine-tune the quantized model by fixing the remaining parameters (e.g., integer weights (714), bias, embedding layer, etc.) except for the scaling factor (712) of the linear layer of the quantized model, and updating only the scaling factor (712). This will be described in more detail below with reference to FIG. 8.

[0071] FIG. 8 is a diagram illustrating an example of a method for fine-tuning a quantized model (800) according to another embodiment of the present disclosure. As described above, the information processing system can fine-tune the quantized model (800) by fixing the remaining parameters (e.g., integer weights (820), bias, embedding layer, etc.) except for the scaling factor (830) of the linear layer (810) of the quantized model (800) and updating only the scaling factor (830). For example, by using learning data associated with a first target task, the integer weights ( )(820) and only the scaling factor (830) can be updated to fine-tune the quantized model (800) to be suitable for the first target task. As a specific example, the learning input data included in the learning data associated with the first target task can be input into the quantized model (800) to perform a forward propagation process, the total loss (840) can be calculated, and only the scaling factor (830) can be updated through a backpropagation process based on the calculated total loss (840). The fine-tuned weights can be expressed as in the following mathematical expression 5.

[0072]

[0073] Here, is a process of fine-tuning for target tasks. Represents the gradient update of .

[0074] During the fine-tuning process for each target task, Value and Since the values ​​are fixed, to manage the model for each target task, Since only the values ​​need to be managed separately, management convenience can be increased. For example, by keeping the integer weight (820) with a relatively large proportion fixed and converting the scaling factor gradient update value (832) for the first target task with a relatively small proportion into the scaling factor gradient update value (834) for the second target task, the first target task can be switched to the second target task, allowing for a quick and easy task switch.

[0075] FIG. 9 is a diagram illustrating an example of comparative evaluation of the performance of a model according to one embodiment of the present disclosure. The illustrated performance comparison table (900) presents the sizes of learnable parameters, weight sizes, and performance scores measured by two language model evaluation means (MNLI, SAMSum) for each of a model generated by a first route that performs fine-tuning followed by quantization and a model generated by a second route that performs fine-tuning followed by quantization (one embodiment of the present invention).

[0076] Referring to the illustrated performance comparison table (900), the model generated in the first route demonstrates a very large learnable parameter size, as fine-tuning is performed on a machine learning model containing a large number of parameters prior to quantization. This suggests that the fine-tuning process requires a significant amount of resources. Furthermore, it can be seen that the model's performance score is significantly reduced during the quantization process after fine-tuning.

[0077] In contrast, the model generated by the second route (one embodiment of the present invention) shows a significant reduction in the size of the learnable parameters due to the parameter-efficient fine-tuning method. This suggests a significant reduction in the resources required for the fine-tuning process. Furthermore, it is confirmed that excellent performance scores are maintained despite the small model size (small weight sizes). Consequently, accurate inference can be performed at a fast inference speed using a small memory footprint.

[0078] FIG. 10 is a diagram illustrating an example of comparing and evaluating the model size, the optimizer size, and the degree of confusion according to one embodiment of the present disclosure. The first graph (1010) represents the model size and the optimizer state size for full fine-tuning, the existing parameter-efficient fine-tuning (PEFT), and the post-quantization fine-tuning according to some embodiments of the present invention, respectively. Referring to the first graph (1010), in the case of full fine-tuning, since updates are performed on all parameters of a large machine learning model, it can be seen that the model size and the optimizer state size are very large. In the case of existing parameter-efficient fine-tuning, since updates are performed on only some parameters of the machine learning model, the optimizer state size is relatively reduced, but the model size remains the same because the model is not compressed. In the case of post-quantization fine-tuning according to some embodiments of the present invention, since both quantization and parameter-efficient fine-tuning are performed, it can be seen that both the model size and the optimizer state size are reduced.

[0079] The second graph (1020) shows the perplexity relative to the model size for the existing parameter-efficient fine-tuning and the fine-tuning after quantization according to an embodiment of the present invention. Perplexity is a measure commonly used to evaluate the predictive performance of a language model, and is a measure that quantifies how difficult it is for a model to predict new or previously unseen data. A lower perplexity value indicates that the model understands and predicts data better, so a lower perplexity value (in the +y direction of the second graph (1020)) indicates better predictive performance of the model. Referring to the second graph (1020), it can be confirmed that the perplexity value relative to the same model size is significantly lower in the case of the present invention compared to the case of the existing parameter-efficient fine-tuning. In other words, it can be confirmed that the model performance relative to the same model size is significantly superior in the case of the present invention compared to the case of the existing parameter-efficient fine-tuning.

[0080] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0081] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0082] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0083] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0084] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0085] When implemented in software, the techniques may be stored on or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

[0086] For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of media. Disk and disc, as used herein, includes compact discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks usually reproduce data magnetically, whereas discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0087] A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside as discrete components in the user terminal.

[0088] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0089] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. A method for compressing and fine-tuning a machine learning model, performed by at least one processor, generating a quantized model by quantizing at least some of the parameters of the pre-trained model; and A step of fine-tuning the quantized model to be suitable for the target task by fixing some of the parameters of the quantized model and updating only some of the remaining parameters of the quantized model using learning data associated with the target task. A method for compressing and fine-tuning a machine learning model, including:

2. In paragraph 1, The step of generating the above quantized model is: A step of converting at least some of the first precision parameters included in the above pre-trained model into a combination of second precision parameters and a scaling factor. Including, A method for compressing and fine-tuning a machine learning model, wherein the second precision is lower than the first precision.

3. In paragraph 2, The above fine-tuning steps are: A step of fine-tuning the quantized model by fixing the second precision parameter among the quantized models and updating only the scaling factor. A method for compressing and fine-tuning a machine learning model, including:

4. In paragraph 2, A method for compressing and fine-tuning a machine learning model, wherein at least some of the parameters of the first precision that are converted into a combination of the parameters of the second precision and the scaling factor are linear transformation weights.

5. In paragraph 2, A method for compressing and fine-tuning a machine learning model, wherein a plurality of parameters among the parameters of the second precision share the same scaling factor in the above quantized model.

6. In paragraph 5, A method for compressing and fine-tuning a machine learning model, wherein, in the above quantized model, multiple weights belonging to the same row of the same weight matrix among the parameters of the second precision share the same scaling factor.

7. In paragraph 2, The above conversion step is, A step of converting each of at least some of the parameters of the first precision included in the above pre-trained model into a combination of a predetermined number of binary values ​​and a scaling factor. A method for compressing and fine-tuning a machine learning model, including:

8. In paragraph 2, The above conversion step is, A step of converting each of at least some of the parameters of the first precision included in the above pre-trained model into a combination of integers and scaling factors. Including, A method for compressing and fine-tuning a machine learning model, wherein the above integer is an integer within a range that can be expressed with a predetermined number of bits.

9. In paragraph 2, In the above quantized model, activation is a method for compressing and fine-tuning a machine learning model that is not quantized to the second precision.

10. In paragraph 1, A method for compressing and fine-tuning a machine learning model, wherein in the above fine-tuning step, the parameters of the bias and embedding layers of the quantized model are fixed.

11. A computer-readable non-transitory recording medium recording commands for executing the method according to paragraph 1 on a computer.

12. As an information processing system, memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, By quantizing at least some of the parameters of a pre-trained model, a quantized model is created, An information processing system comprising instructions for fine-tuning a quantized model to be suitable for a target task by fixing some of the parameters of the quantized model and updating only some of the remaining parameters of the quantized model using training data associated with the target task.

13. In paragraph 12, Generating the above quantized model is: comprising converting at least some of the parameters of the first precision included in the above pre-trained model into a combination of parameters of the second precision and a scaling factor; An information processing system wherein the second precision is lower than the first precision.

14. In paragraph 13, The above fine-tuning is, An information processing system comprising fine-tuning the quantized model by fixing the second precision parameter among the quantized models and updating only the scaling factor.

15. In paragraph 13, An information processing system, wherein at least some of the parameters of the first precision that are converted into a combination of the parameters of the second precision and the scaling factor are linear transformation weights.

16. In paragraph 13, An information processing system, wherein, in the above quantized model, a plurality of parameters among the parameters of the second precision share the same scaling factor.

Images