Method and apparatus for modularized weight-lightening of machine learning model

Abstract

Disclosed are a method and apparatus for modularized weight-lightening of a machine learning model. According to an aspect of the present disclosure, provided is a computer-implemented method for weight-lightening of a module unit of a machine learning model, the method comprising the steps of: loading configuration information of one or more modules having a hierarchical structure including two or more levels, the configuration information including a set value for a lightweight algorithm; and generating an instance for each of the one or more modules on the basis of the configuration information, wherein the configuration information is loaded in the order from the highest level to the lowest level of the hierarchical structure, and the generation of the instance is completed in the order from the lowest level to the highest level of the hierarchical structure.

Description

Method and device for modularizing and lightweighting machine learning models

[0001] The present disclosure relates to a method and device for modularizing and lightweighting a machine learning model.

[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art.

[0003] With the commercialization of large-scale language models, the importance of quantized models for efficient implementation is growing. In line with this trend, various quantization libraries and toolkits are being provided. However, existing open-source quantization libraries require the configuration of the entire model to be predefined based on the quantization algorithm provided by each library, which limits the freedom of model development considering quantization.

[0004] Meanwhile, currently widely used machine learning frameworks are structured to design models at the level of specific computational blocks or modules, a practice particularly evident in the development of artificial neural network models. In this context, the existing approach of establishing the overall model configuration and then simply applying a quantization algorithm is not aligned with the modularized development trend of artificial neural network models. This makes it difficult to leverage existing libraries when developing new quantization algorithms, forcing developers to build a new development environment from scratch before developing the quantization algorithm.

[0005] Furthermore, existing open-source libraries for quantization offer limited customization to individual hardware characteristics. While artificial neural network accelerators have sought to design more optimized hardware beyond traditional standards (e.g., IEEE-754), a simulation environment for this purpose is lacking.

[0006] The present disclosure aims to provide a method and device capable of controlling lightweighting (e.g., quantization or pruning) in units of modules or sub-modules thereof, rather than the entire machine learning model.

[0007] The present disclosure aims to provide a method and device that facilitates the design of a quantization algorithm for research purposes and enables functional verification (functional simulation) prior to hardware design, based on the modularization philosophy of a machine learning framework.

[0008] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] According to one aspect of the present disclosure, a computer-implemented method for lightweighting a module unit of a machine learning model is provided, comprising: a process of loading configuration information of one or more modules having a hierarchical structure including two or more levels, wherein the configuration information includes settings for a quantization algorithm; and a process of generating an instance for each of the one or more modules based on the configuration information, wherein the configuration information is loaded in order from the highest level to the lowest level of the hierarchical structure, and the generation of the instances is completed in order from the lowest level to the highest level of the hierarchical structure.

[0010] According to another aspect of the present disclosure, there is provided a device for lightweighting a module unit of a machine learning model, comprising: at least one memory for storing instructions; and at least one processor, wherein the at least one processor performs a process of loading configuration information of one or more modules having a hierarchical structure including two or more levels by executing the instructions, the configuration information including a setting value for a quantization algorithm; and a process of generating an instance for each of the one or more modules based on the configuration information, wherein the configuration information is loaded in order from the highest level to the lowest level of the hierarchical structure, and the generation of the instance is completed in order from the lowest level to the highest level of the hierarchical structure.

[0011] According to another aspect of the present disclosure, there is provided a computer-readable recording medium having stored thereon a command, which, when executed by the computer, causes the computer to perform the steps of: loading configuration information of one or more modules having a hierarchical structure including two or more levels, the configuration information including settings for a quantization algorithm; and generating an instance for each of the one or more modules based on the configuration information, wherein the configuration information is loaded in order from the highest level to the lowest level of the hierarchical structure, and the generation of the instances is completed in order from the lowest level to the highest level of the hierarchical structure.

[0012] According to an embodiment of the present disclosure, there is an effect in that weight reduction can be controlled by module or sub-module unit rather than the entire machine learning model.

[0013] According to the embodiment of the present disclosure, there is an effect of facilitating the design of a quantization algorithm for research purposes.

[0014] According to an embodiment of the present disclosure, there is an effect that functional verification of hardware is possible prior to designing artificial intelligence accelerator hardware.

[0015] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0016] FIG. 1 is a schematic diagram illustrating the software architecture of a modular lightweight toolkit according to one embodiment of the present disclosure.

[0017] FIG. 2 is an exemplary diagram schematically illustrating the configuration of a layer module according to one embodiment of the present disclosure.

[0018] FIG. 3 is an exemplary diagram schematically illustrating the relationship between classes for implementing modules according to one embodiment of the present disclosure.

[0019] FIG. 4 is an exemplary diagram schematically illustrating the relationship between classes for managing the configuration of modules according to one embodiment of the present disclosure.

[0020] FIG. 5 is an exemplary diagram illustrating a process of instantiating a layer module and its sub-modules according to one embodiment of the present disclosure.

[0021] FIG. 6 is an exemplary diagram illustrating an application example of a modularized lightweight toolkit according to one embodiment of the present disclosure.

[0022] FIG. 7 is a flowchart illustrating a computer implementation method for lightweighting a module unit according to one embodiment of the present disclosure.

[0023] FIG. 8 is a block diagram schematically illustrating an exemplary computing device that can be used to implement a method or device according to the present disclosure.

[0024] Hereinafter, some embodiments of the present disclosure will be described in detail using exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, when describing the present disclosure, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present disclosure.

[0025] In describing components of embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b) may be used. These symbols are only for distinguishing the components from other components, and the nature, order, or sequence of the components are not limited by the symbols. When a part in the specification is said to "include" or "have" a component, this does not mean that other components are excluded, but rather that other components may be included, unless explicitly stated otherwise.

[0026] The detailed description set forth below, together with the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure and is not intended to represent the only embodiments in which the present disclosure may be practiced.

[0027] FIG. 1 is a schematic diagram illustrating the software architecture of a modular lightweight toolkit according to one embodiment of the present disclosure.

[0028] This disclosure presents a software architecture that allows lightweighting to be controlled at the module (or sub-module) level, rather than at the model level. Lightweighting may include, for example, quantization or pruning. While the following description focuses on quantization, it should be noted that this disclosure is not limited to this method and can be applied to various other lightweighting methods.

[0029] In machine learning frameworks, the term "module" refers to the smallest building block that is built within a model. Most machine learning models are implemented in modules, with a high-level programming language (such as PyTorch) wrapping a low-level programming language (such as C++ or CUDA) for acceleration. For example, the torch.nn.Linear or torch.functional.linear modules provided by PyTorch can be implemented by wrapping NVIDIA cuDNN (CUDA Deep Neural Network library) for acceleration on NVIDIA GPUs.

[0030] Referring to FIG. 1, a machine learning model may form a hierarchical structure based on a module (hereinafter, layer module (10)) and its sub-modules, block module (100) and element module (1000-1060). That is, in the hierarchical structure presented in the present disclosure, the layer module (10) may correspond to a root module of the highest layer, and the element modules (1000-1060) may correspond to leaf modules of the lowest layer.

[0031] A layer module (10) may refer to a functional unit that constitutes a machine learning model. For example, a linear layer, a sigmoid layer, and / or a convolutional layer may be implemented as a layer module. The layer module (10) may be composed of a combination of block modules (100).

[0032] A block module (100) may refer to a unit in which tensor operations are performed. The block module (100) may correspond to an operation node and / or an edge of a graph (e.g., a computation graph) representing a model. All tensors can be tracked through the block module (100), which is useful for calculating and tagging quantization parameters for activation values ​​essential for creating a calibration table.

[0033] Element modules (1000-1060) can be classified into four types: kernel (1000), calibrator (1020), estimator (1040), and collector (1060).

[0034] The kernel (1000) can perform operations on tensors. For example, operations such as addition or multiplication can be implemented as a single kernel. In general, operations performed in the kernel are not directly implemented in a high-level programming language due to efficiency or operation support issues, but are implemented in a form that wraps code written in a low-level programming language. The kernel (1000) can transform (or quantize) the input tensor (x) based on a quantization parameter calculated by a regulator (1020) to be described later.

[0035] The regulator (1020) is a unit that calculates parameters (e.g., quantization parameters) required for weight reduction. For example, in integer-based quantization, the regulator (1020) calculates scale and offset for integerization, and may store the calculated quantization parameters in a memory buffer within the regulator (1020). As another example, in floating-point-based quantization, the regulator (1020) may support and calculate an additional exponent bias. The specific function of the regulator (1020) may vary depending on the applied weight reduction method (e.g., quantization or pruning).

[0036] The estimator (1040) obtains statistics of the input tensor (x) that the adjuster (1020) needs to calculate the lightweight parameter. For example, if the adjuster (1020) calculates the quantization parameter based on the largest absolute maximum value of the input tensor (x), the estimator (1040) can calculate the largest absolute maximum value from the input tensor (x).

[0037] The collector (1060) can perform data preprocessing on the input tensor (x) so that the estimator (1040) can obtain statistics. For example, when statistics need to be calculated from sorted data, the collector (1060) can convert the data into ordered statistics.

[0038] FIG. 2 is an exemplary diagram schematically illustrating the configuration of a layer module according to one embodiment of the present disclosure.

[0039] Figure 2 illustrates an example of decomposing the operations of a fully connected layer, which is a layer module unit, into block module units. Hereinafter, "layer module" may be abbreviated as "layer," "block module" may be abbreviated as "block," and "element module" may be abbreviated as "element."

[0040] A fully connected layer (20) may include all or part of an input block (200), a weight block (210), a bias block (220), an MVM (Matrix-Vector Multiplication) block (230), a bias addition block (240), an output requantization block (250), and an output quantization block (260).

[0041] The input tensor is converted into a quantized tensor through the input block (200). At this time, the strategy for quantizing the input tensor is determined by the elements within the input block (200). Similarly, the weight block (210) and the bias block (220) may include operations for quantizing the weights and biases, which are parameters of a fully connected layer. At this time, the strategy for quantizing the weight tensor may be determined by the elements within the weight block (210), and the strategy for quantizing the bias tensor may be determined by the elements within the bias block (220). That is, unlike the conventional method that requires setting the configuration for the entire model, in the present disclosure, the quantization strategy can be modified and reflected for each block in the model source code.

[0042] The kernels of the MVM block (230) and the bias addition block (240) can be implemented by simulating the computational units of customized hardware. In addition, any additional parameters required can be calculated by the regulator within each block.

[0043] The output quantization block (260) can quantize the output value of the operation of the fully connected layer (20). Meanwhile, in the case of complete integer quantization, a requantization technique is mainly used, and for this purpose, an output requantization block (250) can be provided in front of the output quantization block (260). The output requantization block (250) can calculate the value of a quantization parameter for requantization and apply it to the requantization process. In the calibration process for requantization, the output requantization block (250) can fetch a scale buffered in the output quantization block (260).

[0044] Below, we describe an example of implementing hierarchically designed modules using an object-oriented programming language.

[0045] FIG. 3 is an exemplary diagram schematically illustrating the relationship between classes for implementing modules according to one embodiment of the present disclosure.

[0046] Referring to FIG. 3, hierarchically designed layers, blocks, and elements (300, 320, and 340) (more specifically, classes for implementing them) can all have a structure in which they inherit a single top-level abstract module (30) (or abstract class). Accordingly, layers and their sub-modules can be consistently controlled through the top-level abstract module (30). That is, when control over the entire model is required, the relevant contents can be implemented in the top-level abstract module (30) to apply consistent rules.

[0047] The parent module (or parent class) of the top-level abstract module (30) may be an abstract module provided by PyTorch (e.g., torch.nn.Module). Accordingly, the functions of the abstract modules of PyTorch (e.g., automatic differentiation, buffer, hook, etc.) can be used in layers, blocks, and elements (300, 320, and 340), and the source code of existing models developed based on PyTorch can be utilized almost as is. TensorFlow, another machine learning framework, also has a top-level abstract module, so the classes of each module can be implemented in a similar manner. In machine learning frameworks that pursue modularity and whose underlying language is an object-oriented language, the modularization method is mostly applicable.

[0048] For the efficiency of implementation of each module class, classes for structurally identical modules can have a shared class as a parent class. That is, a set of specific layer classes (or block classes, or element classes) with the same operational structure can have a single shared layer class (or shared block class, or shared element class) as a parent class. For example, separate classes can be defined for a fully connected layer (20) according to an implementation method (e.g., a quantization method, etc.), and these can inherit a single fully connected layer class (300). As another example, each class of an input block (200), a weight block (210), a bias block (220), and / or an output quantization block (260) within a fully connected layer (20) can inherit a single quantization block class (320).

[0049] FIG. 4 is an exemplary diagram schematically illustrating the relationship between classes for managing the configuration of modules according to one embodiment of the present disclosure.

[0050] In order to set the configuration on a module-by-module basis, a class (hereinafter, configuration class) that defines the configuration of each module may be used. For example, a layer configuration class (40, 400, and 4000) for a specific layer may include configuration information about blocks constituting the layer. As another example, a block configuration class (42, 420, and 4200) for a specific block may include configuration information about elements constituting the layer. As yet another example, an element configuration class (44, 440, and 4400) for a specific element may define arguments related to a quantization strategy to be applied to the element.

[0051] For example, in the configuration class for an estimator, the value of the 'axis_to_reduce' argument can be defined. 'axis_to_reduce' can be used to define along which axis of the tensor the quantization will be performed. For example, if the tensor is a 3D vector with width, height, and length each being 2, and 'axis_to_reduce=(0,1)', the quantization parameters (e.g., scale or offset) can be obtained in the width and height directions.

[0052] As another example, the configuration class for the regulator may define a value for the 'enlarge_ratio' argument. 'enlarge_ratio' can be used to define a scale factor to be applied to the values ​​received from the estimator. For example, if 'enlarge_ratio=2.0', the values ​​output by the estimator will be scaled by a factor of two and then passed to the regulator.

[0053] The arguments defined within the element configuration classes (44, 440, and 4400) are not limited to the examples described above, and various additional or alternative arguments may be defined depending on the implementation.

[0054] Composition classes can be defined (or declared) as classes that store constant values ​​using the 'dataclasses.dataclass' decorator supported in Python 3.6 (todo) or later. Layer composition classes (40, 400, and 4000) can have predefined block composition classes (42, 420, and / or 4200) as member variables. Similarly, block composition classes (42, 420, and 4200) can have predefined element composition classes (44, 440, and / or 4400) as member variables.

[0055] For implementation efficiency, basic configuration classes (40, 42 and 44) ​​for the configuration of a specific module may be predefined, and users (or developers) may define new configuration classes (400 to 440 and 4000 to 4400) by adding additional properties through inheritance or changing (or redefining) existing properties to other properties. For example, for a fully connected layer (20), separate layer configuration classes (400) may be defined according to the implementation method (e.g., quantization method, etc.), and these may inherit one basic layer configuration class (40). In addition, users (or developers) may define (or declare) a customized layer configuration class (4000) that inherits the predefined layer configuration class (400), thereby modifying the quantization strategy applied to lower modules.

[0056] FIG. 5 is an exemplary diagram illustrating a process of objectifying a layer module and its sub-modules according to one embodiment of the present disclosure.

[0057] From a model development perspective, models can be constructed by repeatedly using layer-level modules. Therefore, information about the block-level modules that constitute a specific layer can be predefined as a configuration class. Similarly, information about the element-level modules that constitute a specific block can also be predefined as a configuration class.

[0058] Figure 5 shows the process of recursively objectifying sub-modules while objectifying modules in layers based on preset configuration information.

[0059] Module instantiation may include memory allocation for an instance of the module (e.g., creating a new instance using Python's __new__ method) and initialization (e.g., initializing the instance using Python's __init__ method). From this perspective, completion of objectification (or completion of instance creation) may indicate a state in which the instance has been created (i.e., memory allocated) and initialized.

[0060] An initialization function of a specific layer module (hereinafter referred to as a "layer initialization function", 500) may be automatically called when an objectification command (50) for the corresponding layer module is called. The objectification command (50) may be called, for example, using the function notation of the layer class.

[0061] The layer initialization function (500) can load predefined layer configuration information (S502). The predefined layer configuration information may be, for example, a layer configuration class (400 or 4000) (and / or an instance thereof) provided as an argument of an objectification command (50) for a layer. As another example, a layer class in which some properties are pre-initialized by the layer configuration class (400 or 4000) can be dynamically created using partial or a similar function provided by Python. Table 1 illustrates pseudo-code for dynamically creating a partially defined layer class 'CustomizedLayer' using a predefined layer configuration class 'CustomizedLayerConfig'.

[0062] Layer class: # declare layer class @dataclassLayerConfig: # declare layer class configuration @dataclassCustomizedLayerConifg(LayerConfig): # overwrite configuration # pass class as an argumentcustomizedLayer = partialclass(Layer class, CustomizedLayerConifg)

[0063] Referring to Table 1, you can define (or declare) a customized layer configuration class 'CustomizedLayerConifg' that inherits the predefined layer configuration class 'CustomizedLayerConifg', and provide it as an argument to partial or similar functions to initialize some properties of the layer class 'Layer class'.

[0064] The layer initialization function (50) can load block configuration information defined in the layer configuration information (S504). The block configuration information may be, for example, a block configuration class (420 or 4200) (and / or an instance thereof) included as a member variable of the layer configuration class (400 or 4000).

[0065] The layer initialization function (500) can call an objectification command (52) for a block module based on layer configuration information and / or block configuration information (S506). For example, the layer initialization function (500) can identify an objectification command (52) for each block module constituting the layer module based on the block configuration information. For example, the layer initialization function (500) can identify the class name of the block module using 'baseclass' in the block configuration class (420 or 4200). The layer initialization function (50) can provide the block configuration information as an argument to the identified objectification command (52).

[0066] When an objectification command (52) for a specific block module is called, an initialization function of the block module (hereinafter referred to as a 'block initialization function', 520) may be automatically called.

[0067] The block initialization function (520) can reload predefined block configuration information (S522). The predefined block configuration information may be a block configuration class (420 or 4200) (and / or an instance thereof) provided as an argument to the objectification command (52) for the block module in step S506.

[0068] The block initialization function (520) can load element configuration information defined in the block configuration information (S524). The element configuration information may be, for example, an element configuration class (440 or 4400) (and / or an instance thereof) included as a member variable of the block configuration class (420 or 4200).

[0069] The block initialization function (520) can call an objectification command (54) for an element module based on block configuration information and / or element configuration information (S526). For example, the block initialization function (520) can identify an objectification command (54) for each element module constituting the block module based on the element configuration information. For example, the block initialization function (520) can identify the class name of the element module using 'baseclass' in the element configuration class (440 or 4400). The block initialization function (52) can provide the element configuration information as an argument to the identified objectification command (54).

[0070] When an objectification command (54) for a specific element module is called, an initialization function of the element module (hereinafter referred to as the 'element initialization function', 540) may be automatically called.

[0071] The element initialization function (540) can reload predefined element configuration information (S542). The predefined element configuration information may be an element configuration class (440 or 4400) (and / or an instance thereof) provided as an argument to the objectification command (54) for the element module in step S526.

[0072] The element initialization function (540) can initialize an instance of an element module (hereinafter, element instance) based on element configuration information (S548).

[0073] As the creation of an element instance is completed, a reference to that element instance (e.g., a pointer or an abstracted memory address) may be returned to the block initialization function (520).

[0074] The block initialization function (520) can call an objectification command for each of the element configuration classes (440 or 4400) included as a member variable of the block configuration class (420 or 4200) and return an element instance corresponding thereto.

[0075] The block initialization function (520) can initialize an instance of a block module (hereinafter, “block instance”) based on the generated element instances (S528). For example, the block initialization function (520) can assign element instances as member variables of the block instance.

[0076] As the creation of a block instance is completed, a reference to that block instance (e.g., a pointer or an abstracted memory address) may be returned to the layer initialization function (500).

[0077] The layer initialization function (500) can call an objectification command for each of the block configuration classes (420 or 4200) included as a member variable of the layer configuration class (400 or 4000) and return a corresponding block instance.

[0078] The layer initialization function (500) can initialize instances of layer modules (hereinafter, layer instances) based on the generated block instances (S508). For example, the layer initialization function (500) can assign block instances as member variables of the layer instance.

[0079] As described above, according to one embodiment of the present disclosure, the configuration information of each module is initialized in a top-down manner in the order of layer, block, and element, while the objectification of the module can be completed in a bottom-up manner in the order of element, block, and layer. Through this process, an instance of a specific layer module can have instances of block modules, which are sub-modules of the layer module, as member variables. Similarly, an instance of a specific block module can have instances of element modules, which are sub-modules of the block module, as member variables.

[0080] If the proposed toolkit is implemented in Python, the repetitive steps involved in objectifying hierarchical modules can be simplified through decorators, thereby increasing implementation efficiency. Tables 2 and 3 illustrate pseudo-code for decorators that can be applied to layer initialization functions and block initialization functions.

[0081] load_config(LayerConfig, ...) layer_config = LayerConfig(); block_dict = dict(); for block block0= block_dict["block0"] ...

[0082] load_config(BlockConfig, ...) block_config = BlockConfig(); element_dict = dict(); for element ...): element0= element_dict["element0"] ...

[0083] Referring to Tables 2 and 3, a series of processes that reload predefined configuration information for a module, load configuration information for a submodule, and perform objectification commands for the submodule within the initialization function of a specific class can be implemented as a decorator by bundling them into the 'load_config' function. Similarly, functions that perform common functions can be bundled and used in other object-oriented languages. For example, in C++, functions similar to decorators can be implemented by appropriately utilizing templates, etc.

[0084] Below, an application example of a modular lightweight toolkit according to one embodiment of the present disclosure is described.

[0085] Since the kernels provided by CPUs and GPUs follow the IEEE-754 standard, experiments on customized hardware are impossible. In the case of FP8 (Floating Point 8-bit), there is no standard standard, and furthermore, there is no specialized operator, so implementation at the bit level is inevitable. The toolkit proposed in this disclosure can be easily implemented by customizing only the operation part of the kernel (1000) using a low-level programming language. In addition, the number representation method is configured to be managed separately as a Number class, so that when a specific number representation is used, information about the number can be obtained by implementing it in the Number class.

[0086] If additional parameters are required for customized numerical expressions or operators, the adjuster (1020) can be used to obtain the values ​​of the parameters. Furthermore, if statistics are required, the estimator (1040) and collector (1060) can be utilized, thereby facilitating code reuse.

[0087] FIG. 6 is an exemplary diagram illustrating an application example of a modularized lightweight toolkit according to one embodiment of the present disclosure.

[0088] Recently, the use of Model Zoo (600), which provides model source code for various trained models and machine learning frameworks, such as Huggingface, TorchVision, and Timm (PyTorch Image Models), has been increasing. In particular, demand for Model Zoo (600) is increasing for models that require extensive pre-training resources, such as large language models (LLMs).

[0089] A developer can download the source code of a specific model (60) from the model host (600) in a local development environment (610), and perform a simulation of the model (60) on the machine learning framework (620) in which the model (60) was developed. Since the modularized lightweight toolkit (630) according to the present disclosure is developed based on the machine learning framework (620), the source code of the original model (60) received from the model host (600) can be converted into a lightweight model (62) with little modification. The modularized lightweight toolkit (630) may include code that simulates a customized operator or numerical expression of a specific device (640). In addition, it may include some functions of a device-specific compiler (650).

[0090] FIG. 7 is a flowchart illustrating a computer implementation method for lightweighting a module unit according to one embodiment of the present disclosure.

[0091] In process S700, the computing device may load configuration information of one or more modules having a hierarchical structure including two or more levels. The configuration information may include settings for a lightweight algorithm (e.g., quantization or pruning). The configuration information may be loaded in order from the highest level to the lowest level of the hierarchy. For example, the computing device may load first configuration information defined in advance for a first module based on a first initialization function being called for a first module having an arbitrary level of the hierarchy. The computing device may load second configuration information for a second module, which is a sub-module of the first module and defined in the first configuration information. The computing device may call an objectification command for the second module based on the second configuration information. For example, the computing device may identify an objectification command defined for the second module based on the second configuration information and provide the second configuration information as an argument to the identified objectification command. Process S700 may be initiated based on a call to an initialization function of a module having a highest level.

[0092] In step S720, the computing device may create an instance for each of one or more modules based on configuration information. The creation of instances may be completed in order from the lowest level to the highest level of the hierarchy. For example, the computing device may initialize an instance of a third module based on third configuration information for a third module at any level of the hierarchy, and return a reference to the instance of the third module to a fourth initialization function for a fourth module, which is a higher module of the third module. Based on the returned reference, the computing device may assign the instance of the third module to a member variable of the instance of the fourth module. The instance of the fourth module may have instances of all lower modules at the same level as the third module as member variables.

[0093] In steps S700 to S720, one or more modules may be modules constituting a machine learning model and / or sub-modules thereof. The one or more modules may include one or more block modules, which are units that perform a predetermined operation and can independently configure a lightweight algorithm. The one or more modules may further include one or more layer modules composed of a combination of one or more blocks. The one or more layer modules may be root modules corresponding to the highest level of the hierarchical structure. The one or more modules may further include a plurality of element modules defined to perform different functions as sub-modules provided for each of the one or more block modules. The plurality of element modules may be leaf modules corresponding to the lowest level of the hierarchical structure.

[0094] The plurality of element modules may include a kernel configured to perform operations defined for the upper block module. Additionally or alternatively, the plurality of element modules may include an adjuster configured to compute parameters required for weighting the upper block module. The kernel may be configured to generate a weighted output tensor based on the parameters. Additionally or alternatively, the plurality of element modules may include an estimator configured to compute statistics for an input tensor of the upper block module. The adjuster may be configured to compute parameters based on the statistics computed by the estimator. Additionally or alternatively, the plurality of element modules may include a collector configured to perform preprocessing on the input tensor of the upper block module. The estimator may be configured to compute statistics based on the input tensor preprocessed by the collector.

[0095] In step S742, the computing device may feed data for performing lightweighting (e.g., training / calibration data (660) of FIG. 6) to the model. Step S742 may optionally be performed if data for performing lightweighting exists (S740, Yes). In general, most lightweighting algorithms require data feeding.

[0096] In step S760, the computing device can calculate parameters for weight reduction (S760). For example, the controller can calculate quantization parameters according to a predetermined algorithm. Based on the calculated parameters, the computing device can reduce model parameters (e.g., weights, etc.).

[0097] In step S780, the computing device may replace model parameters with lightweight parameters. For example, the computing device may replace the model's full-precision parameters with quantized parameters. The quantized parameters may be expressed as in Mathematical Expression 1.

[0098]

[0099] Here, Wq represents the quantized parameter, w represents the model parameter, s represents the quantization parameter (e.g., scale, offset, etc.), and f(·) represents the quantization function. If other lightweighting methods (e.g., pruning) are applied, the quantization function can be replaced with another function.

[0100] FIG. 8 is a block diagram schematically illustrating an exemplary computing device that can be used to implement a method or device according to the present disclosure.

[0101] The computing device (80) may include some or all of a memory (800), a processor (820), storage (840), an input / output interface (860), and a communication interface (880). The computing device (80) may be a stationary computing device such as a desktop computer, a server, or the like, as well as a mobile computing device such as a laptop computer, a smart phone, or the like. The computing device (80) may also include any specialized hardware accelerator capable of efficiently processing operations for an artificial intelligence model. For example, the computing device (80) may include a graphic processing unit (GPU), a tensor processing unit (TPU), or a neural processing unit (NPU).

[0102] The memory (800) may store a program that causes the processor (820) to perform a method or operation according to various embodiments of the present disclosure. For example, the program may include a plurality of instructions executable by the processor (820), and the above-described method or operation may be performed by executing the plurality of instructions by the processor (820). The memory (800) may be a single memory or a plurality of memories. In this case, information required to perform the method or operation according to various embodiments of the present disclosure may be stored in a single memory or may be divided and stored in a plurality of memories. When the memory (800) is composed of a plurality of memories, the plurality of memories may be physically separated. The memory (800) may include at least one of a volatile memory and a non-volatile memory. The volatile memory includes a static random access memory (SRAM) or a dynamic random access memory (DRAM), and the non-volatile memory includes a flash memory.

[0103] The processor (820) may include at least one core capable of executing at least one instruction. The processor (820) may execute instructions stored in the memory (800). The processor (820) may be a single processor or multiple processors.

[0104] Storage (840) maintains stored data even when power supplied to the computing device (80) is cut off. For example, storage (840) may include non-volatile memory, or may include storage media such as magnetic tape, optical disk, or magnetic disk. A program stored in storage (840) may be loaded into memory (800) before being executed by processor (820). Storage (840) may store a file written in a programming language, and a program generated from the file by a compiler or the like may be loaded into memory (800). Storage (840) may store data to be processed by processor (820) and / or data processed by processor (820).

[0105] The input / output interface (860) may provide an interface with an input device such as a keyboard, mouse, etc. and / or an output device such as a display device, printer, etc. A user may trigger execution of a program by the processor (820) through an input device and / or check the processing result of the processor (820) through an output device.

[0106] The communication interface (880) may provide access to an external network. The computing device (80) may communicate with other devices via the communication interface (880).

[0107] Each component of the device or method according to the present invention may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0108] Various implementations of the systems and techniques described herein may be implemented as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations of one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor (which may be a special purpose processor or a general purpose processor) coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device. Computer programs (also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable recording medium."

[0109] A computer-readable recording medium includes any type of recording device that stores data that can be read by a computer system. Such a computer-readable recording medium may be a non-volatile or non-transitory medium such as a ROM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, magneto-optical disk, storage device, and may further include a transitory medium such as a data transmission medium. Furthermore, the computer-readable recording medium may be distributed across network-connected computer systems, so that computer-readable code can be stored and executed in a distributed manner.

[0110] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0111] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0112]

[0113] Statement regarding sponsored research or development

[0114] This invention is the result of research conducted with the support of the National IT Industry Promotion Agency (NIPA) funded by the Ministry of Science and ICT (Project No.: 2021-0-00310, Project Name: Development of a Server-Use SW Framework for Improving Artificial Intelligence Learning / Inference Efficiency).

[0115]

[0116] CROSS-REFERENCE TO RELATED APPLICATION

[0117] This patent application claims priority to Korean Patent Application No. 10-2024-0087754, filed July 3, 2024, which is incorporated herein by reference in its entirety.

Claims

1. A computer implementation method for lightweighting the module units of a machine learning model. A process of loading configuration information of one or more modules having a hierarchical structure including two or more levels, wherein the configuration information includes settings for a lightweight algorithm; and A process for creating an instance for each of the one or more modules based on the above configuration information, The above configuration information is loaded in order from the highest level to the lowest level of the hierarchy, A method in which the creation of the above instances is completed in the order from the lowest level to the highest level of the above hierarchy.

2. In paragraph 1, The above lightweight algorithm is a method including a quantization algorithm or a pruning algorithm.

3. In paragraph 1, One or more of the above modules, A method comprising one or more block modules that perform a predetermined operation and are independently configurable units for the lightweight algorithm.

4. In paragraph 3, The one or more modules further include one or more layer modules composed of a combination of the one or more blocks, A method wherein the one or more layer modules are root modules corresponding to the highest level of the hierarchical structure.

5. In paragraph 3, The one or more modules further include a plurality of element modules defined to perform different functions as sub-modules provided for each of the one or more block modules, A method wherein the above plurality of element modules are leaf modules corresponding to the lowest level of the hierarchical structure.

6. In paragraph 5, The above multiple element modules are, A kernel configured to perform operations defined for the upper block module; A calibrator configured to calculate parameters required for weight reduction of the upper block module; An estimator that calculates statistics for the input tensor of the upper block module; and A collector that performs preprocessing on the input tensor of the upper block module. A method comprising:

7. In paragraph 6, The above estimator is configured to calculate the statistics based on the input tensor preprocessed by the collector, A method wherein the above regulator is configured to calculate the parameter based on the above statistics.

8. In paragraph 6, A method wherein the kernel is configured to generate a lightweight output tensor based on the parameters.

9. In paragraph 1, A method in which the above loading process is initiated based on calling an initialization function of a module having the top level.

10. In paragraph 1, The above loading process is, A process of loading predefined first configuration information for a first module based on a first initialization function being called for a first module having any level of the above hierarchy; and A process of loading second configuration information for a second module, which is a sub-module of the first module, defined in the first configuration information; and A process of calling an objectification command for the second module based on the second configuration information A method comprising:

11. In paragraph 10, The process of calling the above objectification command is: A process of identifying an objectification command defined for the second module based on the second configuration information; and A process of providing the second configuration information as an argument to the objectification command identified above. A method comprising:

12. In paragraph 1, The above generating process is, A process of initializing an instance of a third module based on third configuration information for a third module having any level of the above hierarchical structure; A process of returning a reference to an instance of the third module to a fourth initialization function for the fourth module, which is a superior module of the third module; and A process of assigning an instance of the third module to a member variable of an instance of the fourth module based on the returned reference. A method comprising:

13. In paragraph 12, A method in which an instance of the fourth module has instances of all sub-modules, each of which has the same level as the third module, as member variables.

14. As a device for lightweighting the module units of machine learning models, At least one memory for storing instructions; and at least one processor, At least one processor executes the instructions, A process of loading configuration information of one or more modules having a hierarchical structure including two or more levels, wherein the configuration information includes settings for a lightweight algorithm; and A process of creating an instance for each of the one or more modules based on the above configuration information is performed, The above configuration information is loaded in order from the highest level to the lowest level of the hierarchy, The creation of the above instances is completed in the order from the lowest level to the highest level of the above hierarchy.

15. In paragraph 13, One or more of the above modules, One or more block modules that perform a given operation and are independently configurable units for the lightening algorithm; One or more layer modules comprising a combination of one or more of the above blocks; and A plurality of element modules defined to perform different functions as sub-modules provided for each of the above one or more block modules. A device comprising:

16. In paragraph 15, The above multiple element modules are, A kernel configured to perform operations defined for the upper block module; A regulator configured to calculate parameters required for the lightening of the upper block module; An estimator that computes statistics on the input tensor of the upper block module; and A collector that performs preprocessing on the input tensor of the upper block module. A device comprising:

17. In paragraph 16, The above estimator is configured to calculate the statistics based on the input tensor preprocessed by the collector, The above regulator is configured to calculate the parameter based on the above statistics, A device wherein the kernel is configured to generate a lightweight output tensor based on the parameters.

18. In paragraph 13, The above loading process is, A process of loading first configuration information for a first module defined within the first initialization function based on a first initialization function being called for the first module having any level of the hierarchical structure; and A process of loading second configuration information for a second module, which is a sub-module of the first module, defined in the first configuration information; and A process of calling an objectification command for the second module based on the second configuration information A device comprising:

19. In paragraph 13, The above generating process is, A process of creating an instance of a third module having any level of the above hierarchy; A process of returning a reference to an instance of the third module to a fourth initialization function for the fourth module, which is a superior module of the third module; and A process of assigning an instance of the third module to a member variable of an instance of the fourth module based on the returned reference. A device comprising:

20. A computer-readable recording medium having stored thereon a command, wherein the command, when executed by the computer, causes the computer to: A process of loading configuration information of one or more modules having a hierarchical structure including two or more levels, wherein the configuration information includes settings for a lightweight algorithm; and To execute a process of creating an instance for each of the one or more modules based on the above configuration information, The above configuration information is loaded in order from the highest level to the lowest level of the hierarchy, A computer-readable recording medium in which the creation of the above instances is completed in the order from the lowest level to the highest level of the above hierarchy.

Images