Technique for converting artificial intelligence model

Abstract

According to an embodiment of the present disclosure, disclosed is a method for converting a source artificial intelligence model into a target artificial intelligence model for framework conversion, the method being performed by a computing device. The method may comprise the steps of: receiving the source artificial intelligence model executable in a first framework; converting source operators included in the source artificial intelligence model into a source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework; generating a target intermediate representation structure including target modules dependent on a second framework and the nodes independent of the framework by using information on the second framework into which the source artificial intelligence model is to be converted; and converting the target modules of the target intermediate representation structure into target operators executable in the second framework, so as to generate a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into the target operators. The source modules and the target modules are configured as a combination of the nodes, and each of the nodes may correspond to an atomic operator independent of a framework. The representative figure may be figure 3.

Description

Techniques for transforming artificial intelligence models

[0001] The present disclosure relates to artificial intelligence technology, specifically to transforming an artificial intelligence model.

[0002] With the advancement of artificial intelligence technology, AI models are demonstrating innovative achievements in various fields, such as image recognition, natural language processing, and speech recognition. Most of these AI models are designed to perform specific tasks by learning from vast amounts of data, and their performance tends to be proportional to the complexity of the model and the volume of training data. As model complexity increases, the required computational load and memory usage also increase exponentially.

[0003] In particular, as processor processing speeds increase, large-scale artificial intelligence models are emerging, containing billions of parameters. Training and inferencing these models requires vast amounts of computing resources. The use of computing resources can lead to high operational costs in cloud environments and becomes a major constraint that makes model deployment and utilization difficult in resource-constrained environments, such as edge devices.

[0004] To address these problems, research is being conducted on optimization techniques for artificial intelligence models. Optimization techniques may include methods such as compression and quantization. For example, compression can reduce the size of a model and decrease unnecessary computations by removing parameters or connections that do not significantly contribute to the model's performance. Quantization can reduce memory usage and computational load by representing the model's parameters and activation values ​​with a lower bit width. In this regard, EP 4439275 A1 has been disclosed.

[0005] The present disclosure is devised in response to the aforementioned background technology and aims to efficiently achieve performance transformation or performance optimization for an artificial intelligence model.

[0006] Meanwhile, the technical problem that the present disclosure aims to solve is not limited to the technical problem mentioned above, and various technical problems may be included within the scope obvious to a person skilled in the art from the contents described below.

[0007] According to one embodiment of the present disclosure for realizing the aforementioned task, a method for converting a source artificial intelligence model into a target artificial intelligence model for framework conversion is disclosed, which is performed by a computing device. The method may include: receiving the source artificial intelligence model executable in a first framework; converting source operators included in the source artificial intelligence model into a source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework; generating a target intermediate representation structure including target modules dependent on the second framework and nodes independent of the framework, using information about a second framework to which the source artificial intelligence model is to be converted; and generating a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into target operators by converting the target modules of the target intermediate representation structure into target operators executable in the second framework. The source modules and target modules are composed of combinations of the nodes, and each of the nodes can correspond to an atomic operator independent of the framework.

[0008] In one embodiment, the step of converting the source operators into the source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework may include the step of generating a source intermediate representation structure that represents the source artificial intelligence model in terms of the source modules and the nodes by converting each of the source operators constituting the source artificial intelligence model into a source module composed of the nodes independent of the framework. The source modules may correspond to the source operators dependent on the first framework.

[0009] In one embodiment, the method may further include the step of converting the source intermediate representation structure into a universal intermediate representation structure for conversion to a different framework different from the first framework. The step of generating the target intermediate representation structure may include the step of converting the universal intermediate representation structure into a target intermediate representation structure comprising the target modules dependent on the second framework and the nodes independent of the framework, using information about the second framework to which the source artificial intelligence model is to be converted.

[0010] In one embodiment, the general-purpose intermediate representation structure may include a combination of nodes corresponding to a minimum unit atomic operator that is universally applicable to a plurality of frameworks, as an intermediate representation independent of the framework.

[0011] In one embodiment, the step of converting the source intermediate representation structure into the universal intermediate representation structure may generate the universal intermediate representation structure by removing the source modules dependent on the first framework and retaining the nodes independent of the framework on the source intermediate representation structure, or by modifying the source intermediate representation structure so that a single node is included in a single source model.

[0012] In one embodiment, the step of converting the general-purpose intermediate representation structure into the target intermediate representation structure including the target modules dependent on the second framework and the nodes independent of the framework may include the step of grouping the nodes included in the general-purpose intermediate representation structure using information about the second framework to be converted as a grouping condition, and the step of generating the target modules dependent on the second framework according to the result of the grouping. The target modules may correspond to the target operators.

[0013] In one embodiment, the target intermediate representation structure may include a node, a tensor connecting the nodes, a target module comprising one or more nodes that represents information dependent on the second framework, and a tensor connecting the target modules. The source intermediate representation structure may include a node, a tensor connecting the nodes, a source module comprising one or more nodes that represents information dependent on the first framework, and a tensor connecting the source modules. The target intermediate representation structure and the source intermediate representation structure may further include an argument representing a value used for an operator to operate.

[0014] In one embodiment, the source module in the source intermediate representation structure may include: an input tensor, an output tensor, a source operator dependent on the first framework, nodes existing within the source module, an original module at the time of creation of the source intermediate representation structure, and original nodes at the time of creation of the source intermediate representation structure. The target module in the target intermediate representation structure may include: an input tensor, an output tensor, a target operator dependent on the second framework, nodes existing within the target module, an original module at the time of creation of the target intermediate representation structure, and original nodes at the time of creation of the source intermediate representation structure. A node in the source intermediate representation structure or the target intermediate representation structure may include: an input tensor, an output tensor, an atomic operator corresponding to the node, and original nodes at the time of creation of the source intermediate representation structure.

[0015] In one embodiment, the step of converting the source operators into the source intermediate representation structure may include the step of converting the source operators included in the source artificial intelligence model into a combination of the nodes using a lookup table representing the conversion relationship between the source operators of the first framework and the nodes.

[0016] In one embodiment, the step of converting the source operators into the source intermediate representation structure may include the step of converting each of the source operators included in the source artificial intelligence model into the source module, which is a combination of the nodes, and converting the arguments of the source operators into the arguments of the nodes included in the source module, using a lookup table that includes the conversion relationship between the source operators and the nodes of the first framework and the arguments used for the operators to operate in the first framework.

[0017] In one embodiment, the step of converting the source operators into the source intermediate representation structure may include: identifying a combination of nodes that matches the combination of the source operator of the source artificial intelligence model and the first framework using a lookup table that identifies a combination of nodes that matches a combination of the type of framework and the type of operator; and converting each of the source operators constituting the source artificial intelligence model into the source module by configuring the combination of the identified nodes into a single module.

[0018] In one embodiment, the step of generating the target intermediate representation structure may include the step of determining a recombination method of the nodes for compatibility with the second framework using a lookup table representing a transformation relationship between the target operators of the second framework and the nodes, and the step of generating the target modules dependent on the second framework by grouping the nodes using the determined recombination method.

[0019] In one embodiment, the step of generating the target artificial intelligence model may include the step of generating the target artificial intelligence model compatible with the second framework by converting the arguments of the nodes included in the target modules into the arguments of the target operators of the second framework using a lookup table representing the conversion relationship between the second framework and the nodes and the arguments used for the operator to operate in the second framework.

[0020] In one embodiment, the step of converting the source operators into the source intermediate representation structure may include the step of converting the source operators included in the source artificial intelligence model into a combination of the nodes using a lookup table that uses a framework-specific operator as a key and a node corresponding to an atomic operator as a value.

[0021] In one embodiment, the step of generating the target artificial intelligence model may include the step of generating a quantization operator and a dequantization operator corresponding to the second framework using quantization information of a tensor included in the target intermediate representation structure. When converting the target modules of the target intermediate representation structure into target operators executable in the second framework, the quantization operator and the dequantization operator may be added to the target operators.

[0022] In one embodiment, the method may further include a step of identifying the mode of the received source artificial intelligence model. If the received source artificial intelligence model is identified as a first mode in which the model is executed after representing the operation as a graph, a step of converting the received source artificial intelligence model into the source intermediate representation structure may be performed. If the received source artificial intelligence model is identified as a second mode in which the model is executed in operation units without representing the operation as a graph, a model optimization tool may be applied to the source artificial intelligence model, and a step of converting the source artificial intelligence model to which the model optimization tool is applied into the source intermediate representation structure may be performed. The model optimization tool may include at least one of a compressor, a graph optimizer, and a quantizer.

[0023] In one embodiment, the method may further include the step of changing the configuration of the target modules included in the target intermediate representation structure using a model optimization tool comprising at least one of a compressor, a graph optimizer, and a quantizer. The step of generating the target artificial intelligence model may include the step of generating the target artificial intelligence model executable in the second framework by converting the target modules whose configurations in the target intermediate representation structure have been changed into target operators executable in the second framework.

[0024] In one embodiment, when the target artificial intelligence model receives information about the target hardware to be deployed, the information about the target hardware may be stored in the target module and tensor in the target intermediate representation structure. The step of generating the target artificial intelligence model may include the step of generating the target artificial intelligence model by removing a target module or node corresponding to an operator that is not supported by the target hardware using the information about the target hardware in the target intermediate representation structure.

[0025] In one embodiment, when the target artificial intelligence model receives information about the target hardware and target compiler to be deployed, a node supported by the target hardware and the target compiler and a module including the supported node may be added to the target intermediate representation structure, or a node not supported by the target hardware and the target compiler and a module including the unsupported node may be deleted from the target intermediate representation structure.

[0026] In one embodiment, the method may further include a graph optimization step in which, when the target artificial intelligence model receives information of the target hardware to be deployed, the method uses the information of the target hardware to delete a node on the target intermediate representation structure, delete at least one node among the nodes included in the target intermediate representation structure, or move at least one node among the nodes included in the target intermediate representation structure from a first target module to a second target module on the target intermediate representation structure.

[0027] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. When the computer program is executed by a computing device, the computing device is enabled to perform an operation to convert a source artificial intelligence model into a target artificial intelligence model for framework conversion, the operation may include: receiving the source artificial intelligence model executable in a first framework; converting source operators included in the source artificial intelligence model into a source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework; generating a target intermediate representation structure including target modules dependent on the second framework and nodes independent of the framework, using information about a second framework to which the source artificial intelligence model is to be converted; and generating a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into target operators by converting the target modules of the target intermediate representation structure into target operators executable in the second framework. The source modules and target modules are composed of combinations of the nodes, and each of the nodes can correspond to an atomic operator independent of the framework.

[0028] In one embodiment, a computing device for converting a source artificial intelligence model into a target artificial intelligence model for framework conversion is disclosed. The computing device includes at least one processor, and the at least one processor can: receive the source artificial intelligence model executable in a first framework; convert source operators included in the source artificial intelligence model into a source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework; generate a target intermediate representation structure including target modules dependent on the second framework and nodes independent of the framework, using information about a second framework to which the source artificial intelligence model is to be converted; and generate a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into target operators by converting the target modules of the target intermediate representation structure into target operators executable in the second framework. The source modules and the target modules are composed of combinations of the nodes, and each of the nodes may correspond to an atomic operator independent of the framework.

[0029] According to one embodiment of the present disclosure for realizing the aforementioned task, a method for transforming the performance of an artificial intelligence model, performed by a computing device, is disclosed. The method may include: receiving an intermediate representation structure that represents a target artificial intelligence model to be transformed in terms of a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model, wherein the intermediate representation structure has a hierarchical structure in which one or more nodes constitute the module; modifying at least a portion of the intermediate representation structure by applying a model optimization tool to the intermediate representation structure; and outputting a target artificial intelligence model to which the performance transformation has been applied using the modified intermediate representation structure.

[0030] In one embodiment, the step of changing at least a part of the intermediate representation structure may include the step of determining a target for change among nodes within the intermediate representation structure and tensors connecting the nodes by using information of the module of the intermediate representation structure, and the step of changing at least a part of the intermediate representation structure by changing the determined target for change using the model optimization tool.

[0031] In one embodiment, when at least a part of the intermediate representation structure is changed, the information of the module is used as optimization reference information used by the model optimization tool, so that the hierarchical structure of the intermediate representation structure before and after the change of the intermediate representation structure can be maintained.

[0032] In one embodiment, the step of changing at least a part of the intermediate representation structure may be characterized by maintaining the dependency relationship of the module with respect to the target framework before and after the change of the intermediate representation structure by applying the model optimization tool using information of the target framework included in the intermediate representation structure.

[0033] In one embodiment, the model optimization tool may include at least one of: a compressor for lightweighting the target artificial intelligence model, a graph optimizer for increasing the execution efficiency of the target artificial intelligence model, and a quantizer for reducing at least one of the size and computational amount of the target artificial intelligence model. When a plurality of model optimization tools are applied to the intermediate representation structure, the intermediate representation structure to which a first change by one model optimization tool has been applied may be input to another model optimization tool, and a second change may be applied to the intermediate representation structure to which the first change has been applied by the other model optimization tool.

[0034] In one embodiment, the intermediate representation structure may be a graph structure comprising: a node, a module composed of one or more nodes, and a tensor representing the input of the module, the output of the module, a weight input to the module, and an activation input to the module on the intermediate representation structure. Optimization reference information to be used for the operation of the model optimization tool may be stored in the intermediate representation structure. The optimization reference information may include channel axis tracking information, information indicating whether compression is possible, target hardware information, target framework information, and quantization information.

[0035] In one embodiment, depending on the type of model optimization tool, the optimization reference information to be used by the model optimization tool among the optimization reference information stored in the intermediate representation structure may differ.

[0036] In one embodiment, if the model optimization tool is a compressor, first optimization reference information representing a compressible operator among the operators of the target artificial intelligence model within the intermediate representation structure may be used in the compressor. If the model optimization tool is a graph optimizer, second optimization reference information including quantization information for determining whether to change a module of the intermediate representation structure within the intermediate representation structure may be used in the graph optimizer. If the model optimization tool is a quantizer, third optimization reference information including target platform information on which the target artificial intelligence model will be executed within the intermediate representation structure may be used in the quantizer.

[0037] In one embodiment, the step of changing at least a part of the intermediate representation structure may include the step of obtaining tag information specified by a compressible module or a compressible node from the intermediate representation structure and determining a compression configuration value according to a compressor, and the step of changing at least a part of the intermediate representation structure using the compressor based on the tag information and the compression configuration value.

[0038] In one embodiment, when the compression method of the compressor is a pruning method, information about another module connected to the compressible module or information about another node connected to the compressible node is extracted from the intermediate representation structure, and at least a part of the intermediate representation structure may be modified by the compressor based additionally on the extracted information.

[0039] In one embodiment, when the model optimization tool is a graph optimizer, the step of changing at least one part of the intermediate representation structure may include: determining a target template corresponding to a combination pattern of modules on the intermediate representation structure from among pre-stored module combination templates; and changing at least part of the intermediate representation structure in template units representing combinations of modules by applying the target template to the intermediate representation structure.

[0040] In one embodiment, the step of modifying at least one module of the intermediate representation structure using the graph optimizer may further include the step of determining whether to apply the target template to the intermediate representation structure using quantization information stored in the intermediate representation structure.

[0041] In one embodiment, when the model optimization tool is a graph optimizer, the step of modifying at least one part of the intermediate representation structure may include: determining a target template among pre-stored module combination templates that corresponds to a combination pattern of modules and a combination pattern of nodes on the intermediate representation structure; and modifying at least a part of the intermediate representation structure in template units representing combinations of modules and combinations of nodes by applying the target template to the intermediate representation structure.

[0042] In one embodiment, the method may further include a preprocessing step of removing the quantization-related operator from the intermediate representation structure and adding quantization information corresponding to the quantization-related operator to a tensor within the intermediate representation structure, prior to the step of modifying at least a part of the intermediate representation structure when the received intermediate representation structure includes a quantization-related operator.

[0043] In one embodiment, the step of changing at least a part of the intermediate representation structure may include the step of generating a quantized intermediate representation structure by changing at least a part of the intermediate representation structure using a quantizer. The step of generating the quantized intermediate representation structure may include: determining a range of tensor values ​​for the intermediate representation structure based on target platform information on which the target artificial intelligence model is to be executed; determining quantization parameters for quantizing the target artificial intelligence model using the range of tensor values; and generating a quantized intermediate representation structure by changing a tensor within the intermediate representation structure based on the quantization parameters.

[0044] In one embodiment, the target platform information may include target hardware information on which the target artificial intelligence model is to be executed, target compiler information for connecting the target artificial intelligence model to the target hardware, and runtime information indicating the environment of the target hardware on which the target artificial intelligence model is to be executed.

[0045] In one embodiment, the step of generating the quantized intermediate representation structure may: determine a tensor to be quantized within the intermediate representation structure based on module boundary information in the intermediate representation structure, or determine a tensor to be stored in the intermediate representation structure for quantization information.

[0046] In one embodiment, the intermediate representation structure includes a combination of nodes corresponding to a minimum unit atomic operator that is universally applicable to a plurality of frameworks, and when the frameworks of the artificial intelligence models performing the same operation are different, different modules are included in the intermediate representation structure, and even if the frameworks of the artificial intelligence models performing the same operation are different, the same node may be included in the intermediate representation structure.

[0047] In one embodiment, when the intermediate representation structure is cumulatively modified by a plurality of model optimization tools, information on the number of times the model optimization tools have been applied to the intermediate representation structure is stored within the intermediate representation structure, and when the model optimization tool is applied to the intermediate representation structure, the information on the number of times is used to prevent duplicate application of the model optimization tools.

[0048] In one embodiment, when the intermediate representation structure is modified by a plurality of model optimization tools, a target lightweight value defining a lightweighting goal may be stored in the intermediate representation structure, and an accumulated value for the lightweight values ​​applied to the intermediate representation structure by each of the model optimization tools may be stored in the intermediate representation structure. When a model optimization tool is applied to the intermediate representation structure, the target lightweight value for the intermediate representation structure of the model optimization tool may be determined based on a comparison between the accumulated value and the target lightweight value.

[0049] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. The computer-readable storage medium allows the computing device, when executed by the computing device, to perform a method for transforming the performance of an artificial intelligence model, the method may include: receiving an intermediate representation structure that represents a target artificial intelligence model to be transformed as a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model, wherein the intermediate representation structure has a hierarchical structure in which one or more nodes constitute the module; modifying at least a portion of the intermediate representation structure by applying a model optimization tool to the intermediate representation structure; and outputting a target artificial intelligence model to which the performance transformation has been applied using the modified intermediate representation structure.

[0050] In one embodiment, a computing device for transforming the performance of an artificial intelligence model is disclosed. The computing device may include at least one processor. The at least one processor may perform the following operations: receiving an intermediate representation structure that represents a target artificial intelligence model subject to performance transformation as a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model—the intermediate representation structure having a hierarchical structure in which one or more nodes constitute the module—; modifying at least a part of the intermediate representation structure by applying a model optimization tool to the intermediate representation structure; and outputting a target artificial intelligence model to which performance transformation has been applied using the modified intermediate representation structure.

[0051] According to one embodiment of the present disclosure for realizing the aforementioned task, a method for generating an expected inference simulation result of an artificial intelligence model, performed by a computing device, is disclosed. The method may include: receiving an intermediate representation structure that represents a target artificial intelligence model to be simulated as a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model, wherein the intermediate representation structure has a hierarchical structure in which one or more nodes constitute the module; receiving target platform information identifying a target platform on which the target artificial intelligence model is to be executed and input data to be used in the simulation process of the target artificial intelligence model; and using the intermediate representation structure, the target platform information, and the input data, generating an expected inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform.

[0052] In one embodiment, the step of generating the predicted inference simulation result may include: generating at least one of a target platform node and a target platform module supported on the target platform using the received target platform information; changing the intermediate representation structure using at least one of the target platform node and the target platform module; and generating the predicted inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform using the changed intermediate representation structure and the input data.

[0053] In one embodiment, the target platform module may include the quantization node and the dequantization node, having a structure in which the output of the quantization node corresponding to the target platform and the input of the dequantization node corresponding to the target platform are connected to each other.

[0054] In one embodiment, the step of generating the predicted inference simulation result may include: generating a quantization node and an inverse quantization node corresponding to the target platform using the received target platform information, and modifying the intermediate representation structure by adding the quantization node and the inverse quantization node to the intermediate representation structure; and generating the predicted inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform, using the modified intermediate representation structure and the input data. Using the received target platform information, the location where the quantization node and the inverse quantization node are added on the intermediate representation structure may be determined.

[0055] In one embodiment, the step of generating the predicted inference simulation result may include: generating a simulation intermediate representation structure represented by a connection of nodes by maintaining nodes on the intermediate representation structure but removing modules; and generating the predicted inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform, using the simulation intermediate representation structure and the input data.

[0056] In one embodiment, the simulation intermediate representation structure may be a graph structure composed of nodes and internal edges connecting the nodes.

[0057] In one embodiment, the predicted inference simulation result may be generated by performing inference on a node-by-node basis according to the connection order of nodes within the intermediate representation structure.

[0058] In one embodiment, the step of generating the predicted inference simulation result may include: generating a modified intermediate representation structure by adding a quantization node and an inverse-quantization node corresponding to the target platform to the intermediate representation structure using the received target platform information; generating a simulation intermediate representation structure represented by the connection of nodes by maintaining nodes within the modified intermediate representation structure but removing modules; and generating the predicted inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform, using the simulation intermediate representation structure and a simulator with the input data as input.

[0059] In one embodiment, the intermediate representation structure corresponds to a graph structure including a node, a tensor connecting the nodes, a module composed of one or more nodes, and a tensor connecting the modules.

[0060] In one embodiment, the intermediate representation structure may further include a tensor representing the inputs and outputs, weights, and activations of the modules on the intermediate representation structure. The modules of the intermediate representation structure may include a combination of nodes corresponding to a minimum unit atomic operator that is universally applicable to a plurality of frameworks.

[0061] In one embodiment, the step of generating the predicted inference simulation result may include: comparing platform information stored in the intermediate representation structure with the target platform information; if the result of the comparison step is different from the stored platform information and the target platform information, changing a node constituting a module within the intermediate representation structure, a module within the intermediate representation structure, or a tensor within the intermediate representation structure based on the target platform information; and using the intermediate representation structure to which the change is applied and the input data, generating a predicted inference simulation result for the case where the target artificial intelligence model with the input data as input is executed on the target platform without executing the target artificial intelligence model on the target platform.

[0062] In one embodiment, the target platform information may include target hardware information on which the target artificial intelligence model is to be executed, target compiler information for connecting the target artificial intelligence model to the target hardware, and runtime information indicating the environment of the target hardware on which the target artificial intelligence model is to be executed.

[0063] In one embodiment, the step of generating the predicted inference simulation result may include: generating a modified intermediate representation structure by using the received target platform information to add at least one of a node that is supported on the target platform and a module including the supported node to the intermediate representation structure, or deleting a node that is not supported on the target platform and a module including the unsupported node from the intermediate representation structure; and generating the predicted inference simulation result for the case where the target artificial intelligence model having the input data as input is executed on the target platform without executing the target artificial intelligence model on the target platform, using the modified intermediate representation structure and the input data.

[0064] In one embodiment, the method may further include the step of generating an optimized intermediate representation structure by modifying a combination of nodes within the intermediate representation structure using a model optimization tool comprising at least one of a compressor, a graph optimizer, and a quantizer. The step of generating the predicted inference simulation result may include the step of generating an optimized predicted inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform, without executing the target artificial intelligence model on the target platform, using the optimized intermediate representation structure and the input data.

[0065] In one embodiment, the step of generating the optimized intermediate representation structure may delete at least one node among the nodes included in the intermediate representation structure, compress at least one node among the nodes included in the intermediate representation structure, or move at least one node among the nodes included in the intermediate representation structure from a first module to a second module on the intermediate representation structure.

[0066] In one embodiment, the method may further include the step of generating a performance evaluation result for the optimization of the target artificial intelligence model by comparing the optimization prediction inference simulation result generated using the optimized intermediate representation structure with the prediction inference simulation result generated using the intermediate representation structure.

[0067] In one embodiment, the method may further include: converting the intermediate representation structure into the target artificial intelligence model using a model converter; generating an expected inference simulation result or benchmark result when the input data and the target artificial intelligence model are executed on the target platform and on the target framework of the target artificial intelligence model using the converted target artificial intelligence model and the input data; and comparing the expected inference simulation result generated using the intermediate representation structure with the inference simulation result or benchmark result.

[0068] In one embodiment, the method may further include: a step of determining a range of tensor values ​​when input data is input into an intermediate representation structure based on the target platform information; a step of determining quantization parameters for quantizing the target artificial intelligence model using the range of tensor values; and a step of generating a quantized intermediate representation structure by changing a tensor within the intermediate representation structure based on the quantization parameters. The step of generating the predicted inference simulation result may include a step of generating a predicted inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform, using the quantized intermediate representation structure, the target platform information, and the input data.

[0069] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. When the computer program is executed by a computing device, the computing device performs an operation to generate an expected inference simulation result of an artificial intelligence model, wherein the operation may include: receiving an intermediate representation structure that represents a target artificial intelligence model to be simulated as a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model—the intermediate representation structure having a hierarchical structure in which one or more nodes constitute the module—a target platform information identifying a target platform on which the target artificial intelligence model is to be executed and input data to be used in the simulation process of the target artificial intelligence model, and, using the intermediate representation structure, the target platform information and the input data, generating an expected inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform.

[0070] In one embodiment, a computing device for generating an expected inference simulation result of an artificial intelligence model is disclosed. The computing device includes at least one processor, and the processor may perform the following operations: receiving an intermediate representation structure that represents a target artificial intelligence model to be simulated as a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model—the intermediate representation structure having a hierarchical structure in which one or more nodes constitute the module—a target platform information identifying a target platform on which the target artificial intelligence model is to be executed and input data to be used in the simulation process of the target artificial intelligence model; and, using the intermediate representation structure, the target platform information and the input data, generating an expected inference simulation result for the case where the target artificial intelligence model with the input data is executed on the target platform without executing the target artificial intelligence model on the target platform.

[0071] According to some embodiments of the present disclosure, performance optimization and / or performance transformation of an artificial intelligence model can be achieved through the efficient use of computing resources.

[0072] According to some embodiments of the present disclosure, the conversion of an artificial intelligence model can be performed in a manner independent of the framework of the artificial intelligence model.

[0073] According to some embodiments of the present disclosure, performance transformation and / or performance optimization of an artificial intelligence model can be achieved in a manner independent of the framework of the artificial intelligence model.

[0074] According to some embodiments of the present disclosure, performance transformation and / or performance optimization of an artificial intelligence model can be achieved to suit a platform on which the artificial intelligence model is to be executed.

[0075] According to some embodiments of the present disclosure, expected inference simulation results on an artificial intelligence model can be generated in an efficient manner without running the artificial intelligence model on a platform.

[0076] According to some embodiments of the present disclosure, model optimization tools to be used for the optimization and / or transformation of artificial intelligence models can be efficiently arranged and / or applied.

[0077] According to some embodiments of the present disclosure, an artificial intelligence model optimized for a target platform can be generated.

[0078] According to some embodiments of the present disclosure, by presenting a novel intermediate representation structure, a technical effect can be derived in that the transformation, performance optimization, and / or simulation of artificial intelligence models on various platforms and / or various frameworks can be efficiently achieved.

[0079] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0080] Various aspects are now described with reference to the drawings, wherein similar reference numbers are used to collectively refer to similar components. In the following embodiments, for illustrative purposes, a number of specific details are presented to provide a comprehensive understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details. In other examples, known structures and devices are illustrated in block diagram form to facilitate the description of one or more aspects.

[0081] FIG. 1 is a block diagram of a computing device for performing a technique according to one embodiment of the present disclosure.

[0082] FIG. 2 is a schematic diagram showing a network function according to one embodiment of the present disclosure.

[0083] FIG. 3 illustrates an exemplary structural diagram for converting an artificial intelligence model according to one embodiment of the present disclosure.

[0084] FIG. 4 is a flowchart exemplarily illustrating a method for converting a source artificial intelligence model into a target artificial intelligence model according to one embodiment of the present disclosure.

[0085] FIG. 5 illustrates an intermediate representation structure according to one embodiment of the present disclosure.

[0086] FIG. 6 exemplarily illustrates a method in which a source artificial intelligence model is converted into a target artificial intelligence model using an intermediate representation structure according to one embodiment of the present disclosure.

[0087] FIG. 7 exemplarily illustrates how a model optimization tool is applied to an intermediate representation structure during the conversion process of an artificial intelligence model according to one embodiment of the present disclosure.

[0088] Figure 8 illustrates an exemplary converting process with a model preprocessing procedure applied.

[0089] FIG. 9 illustrates an exemplary structural diagram for converting an artificial intelligence model according to one embodiment of the present disclosure.

[0090] FIG. 10 is a flowchart illustrating, exemplarily, a method for outputting a target artificial intelligence model to which performance transformation has been applied.

[0091] FIG. 11 exemplarily illustrates an intermediate representation structure to which channel axis tracking results are applied according to one embodiment of the present disclosure.

[0092] FIG. 12 shows an example in which a compression technique according to one embodiment of the present disclosure is applied to an intermediate representation structure.

[0093] FIG. 13 shows an example in which a compression technique according to one embodiment of the present disclosure is applied to an intermediate representation structure.

[0094] FIG. 14 exemplarily illustrates an intermediate representation structure including tensor information according to one embodiment of the present disclosure.

[0095] FIG. 15 is a flowchart exemplarily illustrating a method for generating expected inference simulation results according to one embodiment of the present disclosure.

[0096] FIG. 16 exemplarily illustrates a process of generating expected inference simulation results using an intermediate representation structure according to one embodiment of the present disclosure.

[0097] FIG. 17 exemplarily illustrates a process of generating expected inference simulation results by considering platform information according to one embodiment of the present disclosure.

[0098] FIG. 18 exemplarily illustrates a process of generating expected inference simulation results by considering platform information and data type information according to one embodiment of the present disclosure.

[0099] FIG. 19 illustrates an example in which a simulator is applied after converting to a target artificial intelligence model according to one embodiment of the present disclosure.

[0100] FIG. 20 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0101] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0102] As used herein, terms such as “manager,” “editor,” and / or “tool” may refer to computer-related components. A component refers to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or an execution of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. A component may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from one component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0103] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the terms "or" and "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0104] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0105] And, the term "at least one of A or B" should be interpreted to mean "a case including only A," "a case including only B," or "a combination of A and B."

[0106] Terms expressed as N, such as the first, second, or third in the present disclosure, are used to distinguish at least one entity. For example, the entities expressed as the first and second may be identical or different from each other.

[0107] An artificial intelligence model in the present disclosure may refer to a set of programs or functions designed with a mathematical and statistical structure to learn patterns from data and automatically perform tasks such as prediction, classification, generation, and / or inference for a given input. An artificial intelligence model can be constructed by modeling the relationship between input and output in the form of a function. Specific examples of artificial intelligence models will be described later in FIG. 2.

[0108] In the present disclosure, a framework may refer to a set of libraries and tools for designing, building, training, evaluating, and deploying artificial intelligence models. A framework may refer to software that enhances the efficiency and reproducibility of model development. A framework may refer to a development environment for building artificial intelligence models. A framework may be used to encompass the execution environment and / or runtime of an artificial intelligence model. A framework defines the structure and behavior of an artificial intelligence model and enables its execution. For example, frameworks may be exemplified by TensorFlow, PyTorch, ONNX, TensorFlow Lite, TensorRT, JAX, and MXNEt. Each of these various frameworks may differ in the structure of operators and the method of operator processing.

[0109] In the present disclosure, transformation or converting of a framework may mean transforming or converting an artificial intelligence model of a specific framework to an artificial intelligence model of another framework. For example, a source artificial intelligence model executable on a first framework may be transformed or converted to a target artificial intelligence model executable on a second framework. In the present disclosure, transformation means modifying a model and may be used in a sense that encompasses conversion. Depending on the implementation, transformation and conversion may be used interchangeably.

[0110] An intermediate representation structure in this disclosure may refer to a standardized representation format used in an intermediate stage to optimize a model's computational graph or structure for a specific execution environment or platform. An intermediate representation structure in this disclosure may consist of a graph corresponding to an artificial intelligence model, a module representing a layer (or operator) defined dependently with respect to a framework, and a node representing the smallest unit of operators constituting the module. In one embodiment, the intermediate representation structure may be configured such that all nodes within the intermediate representation structure (or constituting the artificial intelligence model) are included in one module. In one embodiment, the intermediate representation structure may be configured such that all nodes within the intermediate representation structure are included in one or more modules. In one embodiment, the intermediate representation structure in this disclosure may take the form of a graph. For example, the graph associated with the intermediate representation structure in this disclosure may refer to the result of abstracting the artificial intelligence model based on the connectivity of modules, nodes, and tensors. An intermediate representation structure in this disclosure may refer to an intermediate or temporary structure for modification to be made for execution on various frameworks and / or various platforms of the artificial intelligence model. An intermediate representation structure in this disclosure may include a graph structure. An intermediate representation structure may be created such that nodes independent of the framework and modules, which are sets of nodes, have a hierarchical structure or dependency relationship. The intermediate representation structure of the present disclosure may be configured to include a node corresponding to a framework-independent atomic operator, a module dependent on the framework, and a tensor that connects the nodes and / or the modules. A tensor is a data structure used to represent the inputs, outputs, weights, intermediate values, and / or quantization values ​​of an artificial intelligence model. For example, a tensor may have a multidimensional array structure. For example, a tensor may contain reference information for the transformation or optimization of an artificial intelligence model.For example, a tensor may be stored at an edge connecting nodes, and / or a tensor may be located at an edge connecting modules. For example, a tensor may be located at an input to a module or node, and / or a tensor may be located at an output from a module or node. The intermediate representation structure in this disclosure may include, for example, a Static Graph of a deep learning framework such as TensorFlow or Onnx, and / or a Dynamic Graph in the form of an Eager execution mode of a deep learning framework such as PyTorch. For example, the intermediate representation structure may be generated by receiving an AI model in the form of a Static Graph, or by receiving an AI model in the form of a Dynamic Graph, and / or by receiving an AI model to which model optimization tool(s) have been applied. The intermediate representation structure in this disclosure may refer to a logical structure for representing the functions of an AI model. The intermediate representation structure in this disclosure may include a combination of nodes corresponding to a minimum unit of an atomic operator that is universally applicable to a plurality of frameworks. For example, if the frameworks of artificial intelligence models performing the same operation are different, different modules are included in the intermediate representation structure, and even if the frameworks of artificial intelligence models performing the same operation are different, the same node may be included in the said intermediate representation structure.

[0111] Multiple levels of abstraction may be represented within the intermediate representation structure of the present disclosure. For example, a module within the intermediate representation structure may refer to a component of a first level of abstraction (e.g., high-level abstraction) representing an artificial intelligence model in units of inference functions, and a node within the intermediate representation structure may refer to a component of a second level of abstraction (e.g., low-level abstraction) representing an artificial intelligence model in units of operators or operations that implement the inference functions of the artificial intelligence model. For example, a module within the intermediate representation structure is a component abstracted into a functional block unit defined to handle a specific sub-function during the overall inference process of the artificial intelligence model. For example, a node within the intermediate representation structure is a component abstracted into an individual operator unit used to implement the sub-functions of the artificial intelligence model.

[0112] In one embodiment, the level of abstraction corresponding to a module may be higher than the level of abstraction corresponding to a node. In one embodiment, an intermediate representation structure may be constructed through a combination of components having different levels of abstraction. In one embodiment, a computing device (100) may hierarchically abstract an artificial intelligence model using modules and nodes through an intermediate representation structure. In one embodiment, the computing device (100) may generate an intermediate representation structure by abstracting the artificial intelligence model to a low level. Model optimization tools may be executed on the intermediate representation structure to which the low-level abstraction is applied, and an optimized or transformed (target) artificial intelligence model may be generated by abstracting the intermediate representation structure on which the model optimization tools were executed to a high level.

[0113] The level of abstraction in this disclosure may refer to the extent to which components constituting an artificial intelligence model are expressed as higher-level concepts, either structurally or semantically. The level of abstraction in this disclosure may refer to the extent to which components of an artificial intelligence model are expressed as units of computation or blocks of computation. The level of abstraction in this disclosure may refer to the extent or scope of grouping the components of a model into logical units. The level of abstraction in this disclosure may refer to an indicator for defining whether the representation method of an artificial intelligence model is conceptual (e.g., source code that is easy for humans to understand) or concrete (e.g., machine language that is easy for machines to understand and execute). For example, the level of abstraction may refer to an indicator indicating how concrete (e.g., low-level) or general or conceptual (e.g., high-level) the method of representing or implementing an artificial intelligence model is.

[0114] For example, the higher the level of abstraction, the greater the degree to which it is expressed as a higher-level concept. For instance, high-level abstraction may refer to conceptualizing or defining the components of an artificial intelligence model in terms of its conceptual or functional aspects. For instance, low-level abstraction may refer to conceptualizing or defining the components of an artificial intelligence model in terms of its implementation or operators. For instance, high-level abstraction may have a larger grouping size compared to low-level abstraction in terms of the grouping size of the components of the artificial intelligence model. In one embodiment, nodes defined according to a relatively low-level abstraction may constitute or abstract a module or graph corresponding to a relatively high-level abstraction.

[0115] In additional embodiments, a low level of abstraction refers to a form directly executable on hardware, such as machine code, and a high level of abstraction refers to a conceptual representation that is easy for humans to understand, such as source code or algorithm blueprints. As an additional example, the level of abstraction may be expressed as multiple levels, such as a low level of abstraction (e.g., machine instructions (binary code) and assembly code), a medium level of abstraction (e.g., source code written in programming languages ​​such as Python or C++), and a high level of abstraction (e.g., mathematical models, pseudocode of algorithms, and block diagrams). For example, a compiler can lower the level of abstraction of an artificial intelligence model by converting human-readable source code into low-level machine code or binary code that can be executed by computer hardware.

[0116] The level of abstraction in the present disclosure may be used interchangeably with the unit of representation or the unit of abstraction according to the abstracted result.

[0117] A technique according to one embodiment of the present disclosure can efficiently achieve transformation and / or optimization of an artificial intelligence model by varying the level of abstraction of the artificial intelligence model. A technique according to one embodiment of the present disclosure can efficiently achieve transformation and / or optimization of an artificial intelligence model by combining both low-level abstraction and high-level abstraction of the artificial intelligence model. A technique according to one embodiment of the present disclosure can efficiently achieve transformation and / or optimization of an artificial intelligence model by applying low-level abstraction to the artificial intelligence model to generate an intermediate representation structure of a second level of abstraction, and applying high-level abstraction to the intermediate representation structure to generate an artificial intelligence model of a first level of abstraction.

[0118] In the present disclosure, the performance of an artificial intelligence model may encompass the accuracy, efficiency, resource usage, inference speed, model lightweighting, and / or latency in inference of the artificial intelligence model. In one embodiment, performance may encompass the performance of the model itself and / or the performance when the model is executed on a platform (e.g., memory usage, latency, model accuracy, and / or fan usage). In one embodiment, performance transformation may mean changing the performance of the artificial intelligence model, and performance optimization may be included within the scope of performance transformation. In one embodiment, performance optimization may mean activities and / or actions that improve performance. For example, performance optimization may include compression, graph optimization, and / or quantization of the artificial intelligence model. For example, performance optimization may mean changing or transforming at least a part of the model to improve such performance. For example, performance optimization may include the optimization of accuracy, which indicates how closely the model's prediction results match the actual correct answer. For example, performance optimization may include transforming the model so that it produces the same or better results using less time or fewer computational resources.

[0119] In the present disclosure, "compression of a model" may refer to a process or algorithm that reduces the storage space occupied by an artificial intelligence model and improves computational efficiency by reconstructing or reducing the structure or parameters of an artificial intelligence model or a group partitioned from an artificial intelligence model. For example, a compression algorithm may include a pruning algorithm that removes unimportant weights or neurons, a quantization algorithm that converts floating-point weights into integers of lower precision, and a knowledge distillation algorithm that trains a lightweight student model to mimic the output of a large teacher model.

[0120] In the present disclosure, graph optimization may refer to a process or algorithm that improves the inference performance of an artificial intelligence model by transforming the computational graph of the artificial intelligence model. For example, the computational graph can represent the flow of operations of the model as nodes (computations) and edges (data flows). Graph optimization can modify such a computational graph to allow the artificial intelligence model to operate on a specific platform and / or allow the artificial intelligence model to run more efficiently and quickly (on a specific platform) during execution.

[0121] In the present disclosure, quantization may refer to a process or algorithm for changing or converting the format of weights, activation values, and / or input values ​​of an artificial intelligence model. For example, quantization may convert the format of weights, activation values, and / or input values ​​of an artificial intelligence model into an integer or fixed-point format with a lower number of bits. For example, quantization may refer to an optimization technique that reduces memory usage and improves computation speed during inference of an artificial intelligence model by changing floating-point values ​​(e.g., 32-bit float) into integers (e.g., 8-bit int).

[0122] In the present disclosure, a model optimization tool may refer to a program or software that is applied to or executed on an artificial intelligence model for model optimization and / or model modification. For example, a model optimization tool may include a trainer for training the model, a compressor for compressing the model, a graph optimizer for graph optimization of the model, a quantizer for quantizing the model, and / or a simulator for simulating expected execution results for the model without running the model on a platform. In one embodiment, the model optimization tool may be used interchangeably with a module for transforming at least a part of the model, such as a core module, a model converter, and / or a model optimizer.

[0123] A platform in the present disclosure may include hardware on which an artificial intelligence model is to be executed and / or a compiler that links the artificial intelligence model to the hardware. In one embodiment, the platform may include hardware. In one embodiment, the platform may include a compiler. In one embodiment, the platform may represent a combination of hardware and a compiler. In one embodiment, the platform may be referred to as an execution environment, a deployment target, a system configuration, a runtime environment, and / or an inference platform on which an artificial intelligence model can be executed.

[0124] In the present disclosure, a transformation scenario may refer to a set of one or more model optimization tools. In one embodiment, each of the plurality of transformation scenarios may identify a combination of model optimization tools to be executed on an artificial intelligence model. In one embodiment, the transformation scenario may indicate model optimization tools and their application order or execution order. In one embodiment, the transformation scenario may indicate the execution order, number of executions, functions, and parameters of the model optimization tools. In one embodiment, the transformation scenario may indicate a predetermined execution order to allow the model optimization tools to be executed sequentially. In one embodiment, the plurality of model optimization tools may be arranged according to their execution order within a single transformation scenario. In one embodiment, the plurality of model optimization tools may be arranged according to their execution order within a single transformation scenario, and the execution parameters of each of the model optimization tools may be defined. In one embodiment, the transformation scenario may identify a second combination of model optimization tools for a first combination of an artificial intelligence model and a platform. For example, the transformation scenario may identify what the combination of model optimization tools is according to the combination of the artificial intelligence model and the platform. For example, the transformation scenario can identify the order in which model optimization tools are executed on the artificial intelligence model for a combination of an artificial intelligence model, hardware, and a compiler. For example, the transformation scenario can indicate a predicted execution performance value for the combination of the artificial intelligence model, hardware, and compiler when the model optimization tools are executed on the artificial intelligence model. For example, the transformation scenario can identify identification information of the model optimization tools to be applied to the artificial intelligence model and optimization parameters used in at least one of the model optimization tools for a combination of an artificial intelligence model, hardware, and a compiler.

[0125] In the present disclosure, a scenario database (Database) may refer to a repository for storing, managing, and updating a plurality of transformation scenarios. In one embodiment, a plurality of transformation scenarios are stored in the scenario database (DB). The scenario DB may refer to a database that is searched and used to recommend the execution order of model optimization tools and the parameters of model optimization tools according to an artificial intelligence model, a platform (e.g., a combination of hardware and a compiler), and / or user requirements. For example, the scenario DB may manage combinations of model optimization tools for combinations of artificial intelligence models and platforms in a table-like structure.

[0126] FIG. 1 is a block diagram of a computing device for converting an artificial intelligence model to optimize the performance of an artificial intelligence model according to one embodiment of the present disclosure.

[0127] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100).

[0128] A computing device (100) according to some embodiments of the present disclosure may be a device for converting an artificial intelligence model. A computing device (100) according to some embodiments of the present disclosure may convert an artificial intelligence model into an intermediate representation structure and / or perform operations for various performance conversions on the intermediate representation structure. A computing device (100) according to some embodiments of the present disclosure may perform an operation of converting the intermediate representation structure into a target artificial intelligence model. A computing device (100) according to some embodiments of the present disclosure may perform simulation operations on an artificial intelligence model using the intermediate representation structure. A computing device (100) according to some embodiments of the present disclosure may be a device for converting an artificial intelligence model to optimize the performance of the artificial intelligence model. The computing device (100) may include any type of server and / or any type of device. The server may include any type of computer system or computer device, such as, for example, a microprocessor, a mainframe computer, a digital processor, a portable device, a device controller, a web server, an application server, or a database server.

[0129] In one embodiment, the computing device (100) may include a user device. For example, the computing device (100) may correspond to a user device that receives data resources generated from any server and performs additional information processing. The user device may include a personal computer (PC), notebook, mobile terminal, smartphone, tablet PC, etc. owned by the user.

[0130] The computing device (100) may include a processor (110), memory (130), and a network unit (150).

[0131] The processor (110) may be composed of one or more cores and may include a processor for performing operations related to data processing, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a digital signal processor (DSP), a tensor processing unit (TPU), and a neural processing unit (NPU) of a computing device (100). The processor (110) may read a computer program stored in memory (130) and perform data processing according to embodiments of the present disclosure. That is, a computer program stored in a computer-readable storage medium (e.g., memory (130)) may cause one or more processors (e.g., processor (110)) to perform operations according to embodiments of the present disclosure. Additionally, the processor (110) may implement any component for performing data processing according to embodiments of the present disclosure.

[0132] According to one embodiment of the present disclosure, the processor (110) may perform operations for training a neural network. For example, the processor (110) may perform operations for training a neural network, such as processing input data for training in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation. At least one of the CPU, GPGPU, DSP, TPU, and NPU of the processor (110) may process the training of a network function. For example, the CPU and GPGPU may together process the training of a network function and data classification using the network function. Additionally, in one embodiment of the present disclosure, processors of a plurality of computing devices may be used together to process the training of a network function and data classification using the network function.

[0133] The processor (110) can typically control the overall operation of the computing device (100). For example, the processor (110) can perform steps included in the flowcharts described below. The processor (110) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through components included in the computing device (100) or by running an application stored in memory (130).

[0134] In one embodiment, the computing device (100) may include a System on Chip (SoC). For example, the computing device (100) may include a single chip in which the processor (110), memory (130), and network unit (150) shown in FIG. 1 are all mounted. In this case, the SoC may further include any modules for implementing the operation of the computing device (100), such as an I / O interface, a Power Management IC (PMIC), and a security module. Meanwhile, it will be obvious to a person skilled in the art that the SoC may be referred to as a processor. Therefore, in this embodiment, operations performed by the processor (110) in the present disclosure should be interpreted as being able to be performed on the SoC.

[0135] In one embodiment of the present disclosure, the memory (130) may store any form of information generated or determined by the processor (110) and / or any form of information received by the network unit (150). In one embodiment, the memory (130) may store information regarding an artificial intelligence model, information regarding a platform, information regarding an intermediate representation structure, information regarding a simulation, information regarding a benchmark, information regarding a converting, information regarding a conversion scenario, information regarding a scenario DB and / or information regarding a model optimization tool, etc.

[0136] In one embodiment of the present disclosure, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and / or an optical disk. The computing device (100) may operate in conjunction with web storage that performs the storage function of the memory (130) on the internet. The memory (130) may include an embedded memory mounted inside the processor (110) and / or a memory mounted outside the processor (110) within the computing device (100). The description of the memory described above is merely an example, and the present disclosure is not limited thereto. The memory (130) may be operated by the processor (110).

[0137] A network unit (150) according to one embodiment of the present disclosure may implement any wired or wireless communication network capable of transmitting and receiving any type of data and signals, etc. The technologies described in this specification may be implemented not only by the networks mentioned above but also by other networks.

[0138] FIG. 2 is a diagram showing a network function according to one embodiment of the present disclosure.

[0139] Throughout this specification, artificial intelligence model, artificial intelligence-based model, computational model, machine learning model, deep learning model, neural network, network function, and neural network may be used interchangeably.

[0140] A neural network can be composed of a set of interconnected computational units, which can generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node. The nodes (or neurons) constituting the neural networks may be interconnected by one or more links. In FIG. 2, the term "node" is used to refer to a neuron within an artificial intelligence model, while the term "node" in the intermediate representation structure is used to represent an atomic operator. Nodes in the intermediate representation structure are used to constitute modules and may have a different meaning from the nodes corresponding to neurons in a neural network. While the set of nodes in FIG. 2 may represent a layer or an operator, the nodes in the intermediate representation structure correspond to an indivisible atomic operator.

[0141] In a neural network, one or more nodes connected via links can form relative input and output node relationships. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0142] In a relationship between an input node and an output node connected through a single link, the value of the output node's data can be determined based on the data input to the input node. Here, the link interconnecting the input node and the output node may have a weight. The weight can be variable and can be varied by the user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node's value can be determined based on the values ​​input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.

[0143] As described above, a neural network consists of one or more nodes interconnected through one or more links, forming input-output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​for the links, the two neural networks may be recognized as different from each other.

[0144] A neural network can be composed of a set of one or more nodes. A subset of nodes constituting a neural network can form a layer. Some of the nodes constituting a neural network can form a layer based on their distances from an initial input node. For example, a set of nodes with a distance of n from an initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a way different from that described above. For example, a layer of nodes may be defined by its distance from a final output node.

[0145] In one embodiment of the present disclosure, a set of neurons or nodes may be defined by the expression a layer.

[0146] Initial input nodes may refer to one or more nodes within a neural network to which data is directly input without passing through links in their relationships with other nodes. Alternatively, in terms of link-based relationships between nodes within the neural network, they may refer to nodes that do not have other input nodes connected by links. Similarly, final output nodes may refer to one or more nodes within a neural network that do not have output nodes in their relationships with other nodes. Furthermore, hidden nodes may refer to nodes constituting the neural network that are neither initial input nodes nor final output nodes.

[0147] A neural network according to one embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. Additionally, a neural network according to another embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer as the number of nodes increases as it progresses from the input layer to the hidden layer. Furthermore, a neural network according to yet another embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer as the number of nodes decreases as it progresses from the input layer to the hidden layer. A neural network according to yet another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer is greater than the number of nodes in the output layer.

[0148] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to input and output layers. Using a deep neural network allows for the identification of latent structures in data. Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, Generative Adversarial Networks (GAN), Restricted Boltzmann Machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, etc. The description of deep neural networks described above is merely illustrative and the present disclosure is not limited thereto.

[0149] The artificial intelligence-based model of the present disclosure can be represented by a network structure of any structure described above, including an input layer, a hidden layer, and an output layer.

[0150] A neural network that can be used in an artificial intelligence-based model of the present disclosure may be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, federated learning for distributed deep learning, or incremental learning. Training of a neural network may be a process of applying knowledge to the neural network to perform a specific operation.

[0151] Neural networks can be trained to minimize the error in their output. The training process involves repeatedly inputting training data into the network, calculating the error between the network's output and the target for the training data, and updating the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In supervised learning, training data is used where the correct answer is labeled for each data point (i.e., labeled training data), whereas in unsupervised learning, the correct answer may not be labeled for each training data point. For instance, in the case of supervised learning regarding data classification, the training data may consist of data where each training data point is labeled with a category. Labeled training data is input into the neural network, and the error can be calculated by comparing the network's output (category) with the labels of the training data.

[0152] As another example, in the case of unsupervised learning for data classification, errors can be calculated by comparing the input training data with the neural network output. The calculated errors are backpropagated within the neural network (i.e., from the output layer to the input layer), and this backpropagation updates the connection weights of each node in each layer of the neural network. The amount of change in the connection weights of each updated node can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of errors can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations in the neural network's learning cycle. For instance, a high learning rate can be used during the early stages of training to quickly achieve a certain level of performance and increase efficiency, while a low learning rate can be used during the later stages to improve accuracy.

[0153] In the training of neural networks, the training data is generally a subset of the real-world data (i.e., the data intended to be processed by the trained neural network). Consequently, a training cycle may exist where errors decrease on the training data but increase on the real-world data. Overfitting is a phenomenon where the network learns excessively on the training data, leading to increased errors on the real-world data. For example, a neural network trained on yellow cats might fail to recognize cats when seeing anything other than yellow, which can be considered a type of overfitting. Overfitting can act as a cause for increased errors in machine learning algorithms. Various optimization methods can be used to prevent this overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (which disables some nodes in the network during training), and the use of batch normalization layers can be applied.

[0154] According to one embodiment of the present disclosure, a computer-readable medium storing a data structure is disclosed. The data structure described above may be stored in a memory (130) in the present disclosure, executed by a processor (110), and transmitted and received by a network unit (150).

[0155] A data structure can refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure can refer to the organization of data for solving specific problems (e.g., data retrieval, data storage, data modification in the shortest possible time). A data structure may also be defined by physical or logical relationships between data elements designed to support specific data processing functions. Logical relationships between data elements may include connections between user-defined data elements. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (e.g., a permanent storage device). Specifically, a data structure may include sets of data, relationships between data, and functions or instructions applicable to the data. Through an effectively designed data structure, a computing device can perform operations while minimizing the use of its resources. Specifically, through an effectively designed data structure, a computing device can increase the efficiency of operations, reading, insertion, deletion, comparison, exchange, and retrieval.

[0156] Data structures can be classified into linear and non-linear data structures based on their form. A linear data structure is one where only one piece of data is connected to the next. Linear data structures can include lists, stacks, queues, and deques. A list can refer to a set of data that maintains an internal order. Lists can include linked lists. A linked list is a data structure where data is connected in a line, with each piece of data possessing a pointer. In a linked list, the pointer can contain information regarding the connection to the next or previous data. Depending on its form, a linked list can be represented as a singly linked list, a doubly linked list, or a circular linked list. A stack is a data arrangement structure that allows for restricted access to data. A stack can be a linear data structure where data can be processed (e.g., insertion or deletion) only at one end. Data stored in a stack can be a Last-In, First-Out (LIFO) data structure, meaning that the later an item is entered, the sooner it is retrieved. A queue is a data sequence structure that allows for limited access to data; unlike a stack, it can be a FIFO (First in First Out) data structure where data stored later is retrieved later. A deque is a data structure that can process data at both ends.

[0157] Non-linear data structures can be structures where multiple data are connected after a single piece of data. Non-linear data structures may include graph data structures. A graph data structure can be defined by vertices and edges, and an edge may include a line connecting two different vertices. Graph data structures may include tree data structures. A tree data structure may be a data structure where there is only one path connecting two different vertices among the multiple vertices included in the tree. In other words, it may be a data structure that does not form loops in a graph data structure.

[0158] The data structure may include a neural network. The data structure including the neural network may be stored on a computer-readable medium. The data structure including the neural network may also include data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, loss functions for learning the neural network, etc. The data structure including the neural network may include any of the components disclosed above. That is, the data structure including the neural network may be composed of all or any combination thereof, such as data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. In addition to the aforementioned components, the data structure including the neural network may include any other information that determines the characteristics of the neural network. Additionally, the data structure may include all forms of data used or generated during the computational process of the neural network, and is not limited to the foregoing. The computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. The neural network may be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. The neural network is composed of at least one node.

[0159] A data structure may include data input to a neural network. A data structure including data input to a neural network may be stored on a computer-readable medium. Data input to a neural network may include training data input during the neural network learning process and / or input data input to a neural network after training is complete. Data input to a neural network may include pre-processed data and / or data subject to pre-processing. Pre-processing may include a data processing process for inputting data into a neural network. Accordingly, a data structure may include data subject to pre-processing and data generated by pre-processing. The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0160] The data structure may include weights of the neural network. (In this specification, weights and parameters may be used interchangeably.) The data structure including the weights of the neural network may be stored on a computer-readable medium. The neural network may include multiple weights. The weights may be variable and may be varied by a user or an algorithm to enable the neural network to perform a desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine the data value output from the output node based on values ​​input to the input nodes connected to the output node and weights set on the links corresponding to each input node. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0161] As an example rather than a limitation, weights may include weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Weights that vary during the neural network learning process may include weights at the start of the learning cycle and / or weights that vary during the learning cycle. Weights for which neural network learning is completed may include weights for which the learning cycle is completed. Accordingly, a data structure containing the weights of a neural network may include a data structure containing weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Therefore, the weights and / or combinations of each weight described above are included in the data structure containing the weights of a neural network. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0162] Data structures containing the weights of a neural network may be stored on a computer-readable storage medium (e.g., memory, hard disk) after undergoing a serialization process. Serialization may be a process of converting a data structure into a form that can be stored on the same or different computing devices and later reconstructed for use. A computing device may serialize the data structure to transmit and receive data over a network. A serialized data structure containing the weights of a neural network may be reconstructed on the same or different computing devices through deserialization. Data structures containing the weights of a neural network are not limited to serialization. Furthermore, data structures containing the weights of a neural network may include data structures designed to increase computational efficiency while minimizing the use of computing device resources (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree in non-linear data structures). The foregoing is merely an example and the present disclosure is not limited thereto.

[0163] The data structure may include hyperparameters of the neural network. The data structure including the neural network hyperparameters may be stored on a computer-readable medium. The hyperparameters may be variables that are varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle iterations, weight initialization (e.g., setting the range of weight values ​​subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layers). The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0164] An AI-based model according to one embodiment of the present disclosure may include a Large Language Model (LLM). In the present disclosure, a Large Language Model may refer to an AI-based model trained using a vast amount of training data to perform Natural Language Processing. The Large Language Model may include a Transformer, a Transformer encoder family model, and / or a Transformer decoder family model. A Transformer encoder family model may correspond to an AI model that uses a Transformer encoder structure. A Transformer decoder family model may correspond to an AI model that uses a Transformer decoder structure. For example, the AI-based model of the present disclosure may include a GPT family Large Language Model, a BERT (Bidirectional Encoder Representations from Transformers) family Large Language Model, etc.

[0165] In one embodiment, the transformer may be composed of an encoder that encodes input data and a decoder that decodes the encoded data. The transformer may have a structure that takes a series of input data as input, undergoes encoding and decoding steps, and outputs a series of output data. In one embodiment, the series of input data may be processed into a form that the transformer can compute. The process of processing the series of input data into a form that the transformer can compute may include a tokenizing process and an embedding process. The tokenizing process may refer to the process of dividing the series of input data into tokens of a certain unit. For example, the certain unit may include word units. The embedding process may refer to the process of converting at least one token tokenized from the series of input data into an embedding vector.

[0166] In one embodiment, the transformer can obtain an embedding vector to be input to an encoder by combining a token embedding vector that embeds at least one token corresponding to a series of input data, a segment embedding vector that distinguishes sentences containing the tokens for each token, and a position embedding vector that reflects the position of the token. The encoder family model and the decoder family model of the transformer can also obtain embedding vectors by performing the same method.

[0167] In one embodiment, for the transformer to encode and decode a series of input data, the encoder and decoder within the transformer may utilize an attention algorithm. An attention algorithm may refer to an algorithm that, for a given query, calculates similarity by applying a softmax function to an attention score obtained by matrix multiplying the query with a key, and calculates an attention value for the query by matrix multiplying the calculated similarity with a value.

[0168] In one embodiment, the self-attention algorithm may refer to an attention algorithm that uses a query, a key, and a value generated by multiplying the same embedding vector by a query weight, a key weight, and a value weight, respectively. The cross-attention algorithm may refer to an attention algorithm that uses a query generated by multiplying a first embedding vector by a query weight, and a key and a value generated by multiplying a second embedding vector by a key weight and a value weight, respectively. The query weight, the key weight, and the value weight may be trainable parameters that are updated through the training process of a large-scale language model.

[0169] In one embodiment, the encoder of the transformer may include an embedding layer, a self-attention layer that applies a self-attention algorithm to an embedding vector, a normalization layer, and a feed-forward neural network (FFN). Additionally, the encoder may have a form in which N unit structures including a self-attention layer, a normalization layer, and a feed-forward neural network are connected. The decoder of the transformer may include an embedding layer, a masked self-attention layer, a normalization layer, a cross-attention layer that applies a cross-attention algorithm, and a feed-forward neural network. Additionally, the decoder may have a form in which N unit structures including a masked self-attention layer, a normalization layer, a cross-attention layer, and a feed-forward neural network are connected. The masked self-attention layer may correspond to a layer that calculates an attention value for each of the sequences in which words are sequentially included among a plurality of words included in a series of input data.

[0170] A transformer may include additional components such as a linear layer and a softmax layer, in addition to an encoder and a decoder. The encoder family model and the decoder family model of a transformer may also each include the aforementioned additional components in addition to the encoder and decoder. A method for constructing a transformer using an attention algorithm may include the method disclosed in Vaswani et al., Attention Is All You Need, 2017 NIPS, which is incorporated herein by reference.

[0171] In one embodiment, attention layers such as a self-attention layer, a masked self-attention layer, and a cross-attention layer may correspond to a multi-head attention layer that includes a plurality of attention layers in parallel. The multi-head attention layer may concatenate the attention values ​​output from each of the plurality of attention layers and output an output attention value by matrix multiplying the concatenated matrix by an output weight. The output attention value output from the multi-head attention layer may have the same size as the attention value output from a single attention layer.

[0172] In one embodiment, the transformer may be trained through a Masked Language Model (MLM) process, a Next Sentence Prediction (NSP) process, etc. The MLM process may refer to a training process that predicts a masked word through a series of training data in which some words are masked. The NSP process may refer to a training process that determines whether two sentences are connected sentences in a series of training data containing any two sentences.

[0173] In one embodiment, a large-scale language model can process various data formats, such as natural language text as well as image data, audio data, and video data. The large-scale language model can embed data to convert data of various data formats into a series of computationally operable data. The large-scale language model can process additional data that represents the relative positional or topological relationships between a series of input data. Alternatively, a series of input data may be embedded by additionally reflecting vectors that represent the relative positional or topological relationships between the input data. In one example, the relative positional relationships between a series of input data may include, but are not limited to, word order within a natural language sentence, the relative positional relationships of each segmented image, and the temporal order of segmented audio waveforms. The process of adding information that represents the relative positional or topological relationships between a series of input data may be referred to as positional encoding.

[0174] An example of a large-scale language model for processing image data (Vision Transformer, ViT) is disclosed in Dosovitskiy, et al., AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE, which is incorporated herein by reference.

[0175] An artificial intelligence model according to one embodiment of the present disclosure may include a multimodal large-scale language model. A multimodal large-scale language model may refer to a large-scale language model capable of understanding and processing relationships between different data formats, such as natural language text data, image data, audio data, and video data. A multimodal language model may include a plurality of encoders that encode input data corresponding to each data format. A multimodal language model may be trained to calculate similarity between embedding vectors encoded by encoders of each data format through training data containing data of different data formats, such that similarity between identical pairs is calculated to be higher and similarity between different pairs is calculated to be lower.

[0176] An example of a large-scale multimodal language model that understands and processes the relationship between image data and natural language text data (Contrastive Language-Image Pre-training, CLIP) is disclosed in Alec Radford, et al., *LEARNING TRANSFERABLE VISUAL MODELS FROM NATURAL LANGUAGE SUPERVISION*, which is incorporated herein by reference.

[0177] FIG. 3 illustrates an exemplary structural diagram for converting an artificial intelligence model according to one embodiment of the present disclosure.

[0178] The entities illustrated in FIG. 3 are illustrated as being included in the computing device (100). Depending on the implementation mode, some of the entities of FIG. 3 may be operated by interacting with the computing device (100) from outside the computing device (100). Depending on the implementation mode, additional entities other than the entities of FIG. 3 may also be implemented in a manner in which they are included in the computing device (100) and operated.

[0179] In one embodiment, a computing device (100) may receive a source artificial intelligence model (200). For example, the source artificial intelligence model (200) may refer to an artificial intelligence model that is subject to transformation. For example, the source artificial intelligence model (200) may refer to a model executable on a source framework. For example, the source artificial intelligence model (200) may be subject to transformation into a target artificial intelligence model (300) executable on a target framework. For example, the source artificial intelligence model (200) may refer to an artificial intelligence model received and / or input from a user terminal. For example, the source artificial intelligence model (200) may refer to a model composed of one or more operators or layers executable on a source framework. Operators and layers in the present disclosure may be used interchangeably depending on the implementation mode. For example, the source artificial intelligence model (200) may include framework information that allows the source artificial intelligence model (200) to be executed. For example, the source artificial intelligence model (200) may include identification information of the artificial intelligence model, executable file information of the artificial intelligence model, training dataset information of the artificial intelligence model, execution environment information of the artificial intelligence model, and / or framework information of the artificial intelligence model. For example, a computing device (100) may receive information identifying the source artificial intelligence model (200) or a file of the source artificial intelligence model (200), and may extract training dataset information of the artificial intelligence model, execution environment information of the artificial intelligence model, and / or framework information of the artificial intelligence model using the received information or file. In one embodiment, the computing device (100) may receive information of the source artificial intelligence model (200) and a target framework (a framework different from the source framework) to which the source artificial intelligence model (200) is to be executed from a user terminal. The computing device (100) may convert the source artificial intelligence model (200) into a target artificial intelligence model (300) executable in the target framework.A computing device (100) can convert a source artificial intelligence model (200) into a target artificial intelligence model (300) executable in a target framework by taking into account the constraint(s) included in the platform information (305). For example, the computing device (100) can receive the source artificial intelligence model (200) and the target framework (e.g., PyTorch, ONNX, TFlite, etc.) to which the model is to be converted, where the target framework may represent a framework different from the source framework in which the source artificial intelligence model (200) is executable.

[0180] In an additional embodiment, the computing device (100) may automatically generate recommendation information for candidate platforms that support the target framework in response to receiving the source artificial intelligence model (200) and / or the target framework. In response to user selection input on this recommendation information, the target platform to be transformed among the candidate platforms may be determined. In this embodiment, platform information (305) may be determined in response to user selection input on the recommendation information. In this embodiment, platform information (305) may be automatically determined by the computing device (100) in response to receiving the target framework.

[0181] In one embodiment, a computing device (100) may receive platform information (305). The platform information (305) may include information for identifying hardware and / or information for identifying a compiler. The platform information (305) may define an environment in which a converted target artificial intelligence model (300) is to be executed. The platform information (305) may identify hardware and / or a compiler that is the target for the execution of a source artificial intelligence model (200). The platform information (305) may be received and / or input from a user terminal. In one embodiment, the platform information (305) may include at least one of hardware on which the artificial intelligence model is to be executed, a compiler for connecting the artificial intelligence model to the hardware, and a runtime indicating the environment of the hardware on which the artificial intelligence model is to be executed. The computing device (100) may extract or generate information regarding the hardware and / or the compiler from the platform information (305) to which the source artificial intelligence model (200) is to be executed. For example, information regarding hardware included in platform information (305) may include information regarding, for example, CPU, GPU, NPU, and FPGA. Information regarding hardware may mean any form of information identifying hardware on which an artificial intelligence model can be executed (e.g., to execute the source artificial intelligence model (200)). For example, information regarding hardware may include information related to the physical components of a system and their characteristics, structure, performance, compatibility, etc. For example, information regarding hardware may include at least one of hardware identification information, supported framework information, supported data type information, supported operator information, or supported granularity information. As another example, information regarding hardware may include hardware constraints.Hardware constraints may include at least one of support for each of the operator's data types, support for hardware operation acceleration of the operator, or speed improvement for each of the operator's data types. For example, a data type may be a type of value and its representation method that can be used when storing and processing data in hardware or a programming language. A data type may be a type of data distinguished according to the method of storing and operating on the data. For example, a data type may include integer types, floating-point types, character types, etc. The hardware supports operations for specific data types, and each data type may have different memory sizes and representation ranges. For example, information regarding the compiler included in the platform information (305) may mean any form of information that identifies an entity connecting the artificial intelligence model and the hardware. For example, the platform may be identified and / or managed as a combination unit of hardware and the compiler. For example, a platform may be identified and / or managed as a combination of hardware and a compiler within a computing device (100), and information regarding the related platform may be extracted or generated by the computing device (100) upon receiving platform information (305).

[0182] In one embodiment, the computing device (100) can determine a transformation scenario of the source artificial intelligence model (200) using the source artificial intelligence model (200) and platform information (305). In one embodiment, the scenario manager (310) of the computing device (100) can determine a transformation scenario using the source artificial intelligence model (200) and platform information (305). In one embodiment, the scenario manager (310) can generate a target transformation scenario to be applied to the source artificial intelligence model (200) among a plurality of transformation scenarios using the source artificial intelligence model (200) and platform information (305) (and / or target performance information). In one embodiment, the scenario manager (310) can build a scenario database by generating a plurality of transformation scenarios. The scenario manager (310) may refer to a component (e.g., a software component) that performs a specific function in the computing device (100).

[0183] In one embodiment, a scenario manager (310) of a computing device (100) accesses a scenario database storing a plurality of transformation scenarios and can determine a target transformation scenario to be applied to the source artificial intelligence model (20) among the plurality of transformation scenarios using source artificial intelligence model (200) and platform information (305).

[0184] In one embodiment, each of the plurality of transformation scenarios stored in the scenario database can identify a combination of model optimization tools to be executed on an artificial intelligence model. Each of the plurality of transformation scenarios stored in the scenario DB can identify a second combination of model optimization tools (320a, 320b, 320c, 375, 380, 385 and / or 390) for a first combination of an artificial intelligence model and a platform. For example, a combination of an artificial intelligence model named A, a platform named B, and model optimization tools arranged in the order of C, D, E, and F may be included in a single transformation scenario. The scenario DB may manage the combinations of model optimization tools (320a, 320b, 320c, 375, 380, 385 and / or 390) for combinations of artificial intelligence models and platforms in a structure, for example, in the form of a table. For example, each of the multiple transformation scenarios can identify the order in which model optimization tools (320a, 320b, 320c, 375, 380, 385 and / or 390) are applied to the artificial intelligence model for a first combination of the artificial intelligence model and platform.

[0185] In one embodiment, the computing device (100) may access a scenario DB and input a combination of a source artificial intelligence model (200) and platform information (305) into the scenario DB. The computing device (100) may receive, acquire, or determine a target transformation scenario that matches the combination of the source artificial intelligence model (200) and platform information (305) among a plurality of transformation scenarios. Upon acquiring the target transformation scenario that matches the combination of the source artificial intelligence model (200) and platform information (305), the computing device (100) may receive, acquire, and / or determine identification information of model optimization tools stored within the target transformation scenario and / or the execution order of the model optimization tools. As an example, but not a limitation, the order in which model optimization tools identified in each of the plurality of transformation scenarios are applied to the artificial intelligence model may be determined based on the performance change prediction value by each of the model optimization tools and constraints according to the platform.

[0186] In one embodiment, the scenario DB may include identification information of an artificial intelligence model or identification information of a block of an artificial intelligence model, identification information of a model optimization tool or identification information of an optimization method, and performance information related to the artificial intelligence model to which the model optimization tool or optimization method is applied. The scenario DB may represent the aforementioned information in a table structure.

[0187] In one embodiment, the scenario DB can store and manage a plurality of transformation scenarios. Each of the plurality of transformation scenarios can display a performance prediction value for a first combination of an artificial intelligence model and a platform when model optimization tools are applied to the artificial intelligence model. For example, the performance prediction value may represent the expected execution performance of the artificial intelligence model when the artificial intelligence model to which the model optimization tools are applied is executed on the platform. For example, the performance prediction value may include at least one of the accuracy of the artificial intelligence model, the inference speed of the artificial intelligence model, and the degree of lightweighting of the artificial intelligence model. For example, the scenario DB can manage a plurality of transformation scenarios and the expected execution performance according to the scenario in the form of a table structure.

[0188] In one embodiment, the computing device (100) can calculate model similarity between a source artificial intelligence model (200) and a plurality of candidate artificial intelligence models stored in a scenario DB. For example, model similarity can be determined using identification information of operators (or layers) included in the model and / or connection information of the operators. For example, model similarity can be calculated for each of the source artificial intelligence model (200) and the candidate artificial intelligence models stored in the scenario DB in units of groups (e.g., blocks) of operators included in the model. In one embodiment, model similarity can be determined by converting the model into an intermediate representation structure and calculating similarity between the intermediate representation structures. For example, the computing device (100) can convert the source artificial intelligence model (200) into a graph structure and quantitatively measure model similarity between artificial intelligence models using the pattern of nodes within the graph structure, identification information of the nodes, the number of nodes, identification information of modules, the number of modules and / or the pattern of modules as factors. For example, the computing device (100) can measure sub-similarity between multiple artificial intelligence models in units of operators or blocks constituting the artificial intelligence model and calculate similarity between all models by combining the measured sub-similarities. For example, even if the artificial intelligence models include the same blocks or the same operators, the computing device (100) can measure the similarity between them using cosine similarity of the output value when the same input is input, because the lightweighting performance may differ if the weights are different. In one embodiment, the computing device (100) can preemptively use model similarity to determine the artificial intelligence model most similar to the source artificial intelligence model (200) within the scenario DB and search the scenario DB for a scenario that maps to the hardware, compiler, user environment, and target performance for the determined model.In one embodiment, when the model similarity is lower than the similarity threshold, the computing device (100) determines that it cannot find a model similar to the source artificial intelligence model (200) in the scenario DB and can perform a transformation scenario preparation process to store the source artificial intelligence model (200) as a new model in the scenario DB. In this embodiment, the computing device (100) can determine whether to switch to the transformation scenario preparation process or maintain the transformation scenario execution process based on the result of comparing the model similarity between the source artificial intelligence model (200) and the models in the scenario DB.

[0189] In one embodiment, the computing device (100) can determine a target transformation scenario to be applied to the source artificial intelligence model (200) among a plurality of transformation scenarios stored in the scenario DB by using model similarity, platform information (305), and / or a target performance value specified by the user as conditions. The computing device (100) can perform a search within the scenario DB by using the aforementioned condition(s) as filtering criteria. For example, the computing device (100) can perform a search on the scenario DB by using a combination of model similarity, platform information (305), and target performance value as filtering criteria. Through a search based on such filtering criteria, a target transformation scenario corresponding to the user's intention can be efficiently obtained.

[0190] In one embodiment, the computing device (100) can determine a reference AI model corresponding to the source AI model (200) among the candidate models stored in the scenario DB based on the calculation result of the model similarity for each of the source AI model (200) and the candidate AI models stored in the scenario DB. The computing device (100) can determine a scenario mapped to platform information (305) and / or a target performance value specified by the user as a target transformation scenario based on the reference AI model. For example, the computing device (100) can determine the reference AI model with the highest similarity to the source AI model (200) among the candidate AI models, and use the determined reference AI model as a search criterion to filter a plurality of transformation scenarios including the reference AI model within the scenario DB. The computing device (100) can determine a target transformation scenario, which is the optimal transformation scenario that matches the user's intention, among the filtered transformation scenarios by using platform information (305) and / or user requirements (e.g., target performance value, etc.) as additional filtering conditions.

[0191] In one embodiment, the computing device (100) may additionally receive a target performance value. This target performance value may define a performance value that serves as a standard for the transformation of the source artificial intelligence model (200). The computing device (100) may compare the target performance value with the performance prediction value of each transformation scenario (or each model optimization tool) within the scenario DB and determine the target transformation scenario to be applied to the source artificial intelligence model (200) based on the result of the comparison. For example, the computing device (100) may search for and / or determine the target transformation scenario in a resource-efficient manner on the scenario DB, which is a relational DB with a table structure, by using the identification information of the source artificial intelligence model (200), platform information (305), and the target performance value as search conditions. For example, the computing device (100) may compare the target performance value with each of the performance prediction values ​​(of each transformation scenario) stored in the scenario DB and perform a search based on the combination of the source artificial intelligence model (200) and platform information (305) targeting the transformation scenarios that passed the criteria based on the comparison. In this way, the optimization method for the user's artificial intelligence model can be determined automatically through the efficient use of computing resources, depending on the conditions intended by the user.

[0192] In one embodiment, the computing device (100) can measure model similarity between a plurality of artificial intelligence models registered in the scenario DB and a source artificial intelligence model (200). Model similarity may represent the similarity between combinations of components (e.g., operators (layers)) included in the model. Based on model similarity, hardware information and compiler information included in platform information (305), user environment and / or target performance, a target scenario most similar to the criteria may be determined among a plurality of transformation scenarios stored in the scenario DB. For example, if the user environment is unable to learn or data does not exist, it is determined that the learner cannot be used, and among the transformation scenarios, transformation scenarios in which the learner (320a) exists may be filtered out. For example, if the user environment is not capable of learning or data does not exist, even if the execution mode of the source artificial intelligence model (200) is determined to be the Eager execution mode (315a), a transformation scenario that does not include the Advanced Compressor (320b) and / or the Advanced Quantizer (320c) may be determined as the target transformation scenario.

[0193] In one embodiment, the computing device (100) may vary the method of determining the target transformation scenario according to the execution method or execution mode of the source artificial intelligence model (200). For example, the correlation between the execution method, execution mode, or execution type of the source artificial intelligence model (200) and the method of determining the target transformation scenario may be automatically determined by the computing device (100). For example, the computing device (100) may determine the execution mode of the source artificial intelligence model (200) as one of an eager execution mode (315a) and a graph execution mode (315b). For example, the eager execution mode (315a) may mean a method of executing operations within the artificial intelligence model without converting them into a graph. For example, the graph execution mode (315b) may mean a method of converting operations into a graph before executing the artificial intelligence model. Each of the artificial intelligence models has an execution mode defined for its execution. The computing device (100) can determine what the execution mode of the source artificial intelligence model (200) is by using the identification information of the source artificial intelligence model (200) and / or by parsing the file of the source artificial intelligence model (200) (e.g., analysis of the identification information of operators and the connection structure). For example, the computing device (100) can determine whether the source artificial intelligence model (200) is in an Eager execution mode (315a) or a Graph execution mode (315b) by executing the source artificial intelligence model (200) on a specific platform. For example, if it is confirmed that the source artificial intelligence model (200) executes operators immediately during the execution process, the computing device (100) can classify the source artificial intelligence model as being in an Eager execution mode (315a). For example, if it is confirmed that the source artificial intelligence model adds operators to the computation graph and executes the computation graph all at once during the execution process, the computing device (100) can classify the source artificial intelligence model (200) as being in a Graph execution mode (315a).In an additional example, the computing device (100) can analyze the framework of the source artificial intelligence model (200) (e.g., source framework) and classify the source artificial intelligence model (200) into an Eager execution mode (315a) if the analyzed source framework belongs to a first group of frameworks and into a Graph execution mode (315b) if it belongs to a second group of frameworks.

[0194] In one embodiment, the computing device (100) receives a model and / or intermediate representation corresponding to various frameworks, such as PyTorch, ONNX, and TFLite, as input, and uses a scenario manager (310) to determine a scenario for sequentially executing model optimization tools and execute the model optimization tools accordingly. By example, not limited to, the model optimization tools may include a learner (320a), a compressor (320b), and a quantizer (320d) that run in eager execution mode (e.g., PyTorch). By example, not limited to, the model optimization tools may include an intermediate representation manager (330), a compressor (375), a quantizer (380), a graph optimizer (385), and a simulator (390) that run in graph execution mode. Graph execution mode is a mode that is represented and executed as a graph without operating in eager execution mode, such as the PyTorch framework. If the graph has a dynamic graph, it can be converted to a static graph via Dynamic to static (325). For example, frameworks such as ONNX and TFLite can be represented as static graphs.

[0195] In one embodiment, the computing device (100) can determine filtered candidate transformation scenarios including the determined execution mode among a plurality of transformation scenarios stored in a scenario DB by using the determined execution mode of the source artificial intelligence model (200) as a filtering criterion. The computing device (100) can determine a target transformation scenario to be applied to the source artificial intelligence model (200) among the filtered candidate transformation scenarios. Since this method can reduce the search space of the scenario DB during the process of determining the target transformation scenario, the technique according to one embodiment of the present disclosure can achieve the technical effect of determining the execution order of model optimization tools suitable for user requirements while efficiently using computing resources.

[0196] In one embodiment, the computing device (100) may determine to use a different model optimization tool or a set of different model optimization tools depending on the execution mode of the source artificial intelligence model (200). A correlation may be established between the execution mode of the source artificial intelligence model (200) and the combination of model optimization tools to be executed on the source artificial intelligence model (200).

[0197] For example, the computing device (100) may determine model optimization tool(s) to execute or apply to the source artificial intelligence model (200) within the range of a model optimization tool set including a learner (320a), a compressor (320b), and a quantizer (320c) for the source artificial intelligence model (200) when the source artificial intelligence model (200) is determined to be in an Eager execution mode (315a). For example, a model optimization tool set including a learner (320a), a compressor (320b), and a quantizer (320c) may be used when the execution mode of the source artificial intelligence model (200) is determined to be an Eager execution mode (315a). For example, the learner (320a) may operate to train the source artificial intelligence model (200). For example, the compressor (320b) may mean an Advanced Compressor. For example, the quantizer (320c) may mean an Advanced Quantizer. For example, the learner (320a), compressor (320b), and / or quantizer (320c) can perform learning, compression, and / or quantization of the artificial intelligence model through runtime operations rather than an intermediate representation structure. For example, the learner (320a), compressor (320b), and / or quantizer (320c) can perform learning, compression, and / or quantization of the artificial intelligence model through runtime operations rather than a graph structure.

[0198] In one embodiment, the computing device (100) may pass the model to an intermediate representation manager (330) by performing a Dynamic to Static conversion on the output from the learner (320a), the compressor (320b), and / or the quantizer (320c). For example, the Dynamic to Static conversion may mean converting a model written or developed in Eager execution mode (e.g., dynamic execution mode) to Graph execution mode (e.g., static graph execution mode). For example, the computing device (100) may trace the execution path of the code in Eager execution mode and use the trace result of the execution path to generate a computational graph structure corresponding to the Graph execution mode.

[0199] In one embodiment, the intermediate representation structure may include a Static Graph of a deep learning framework such as TensorFlow or Onnx, and / or a Dynamic Graph in the form of an Eager execution mode of a deep learning framework such as PyTorch. For example, a computing device (100) may receive an artificial intelligence model in the form of a Static Graph and convert it into an intermediate representation structure (370). For example, a computing device (100) may receive an artificial intelligence model in the form of a Dynamic Graph and convert it into an intermediate representation structure (370). A computing device (100) may receive an artificial intelligence model to which model optimization tool(s) (320a, 320b, and 320c) have been applied and generate it into an intermediate representation structure (370).

[0200] In an additional embodiment, the computing device (100) may change the source artificial intelligence model (200) to a Graph execution mode without the application of a learner (320a), a compressor (320b), and / or a quantizer (320c) by applying a Dynamic to Static transformation to the source artificial intelligence model (200) when the source artificial intelligence model (200) is determined to be in an Eager execution mode (315a).

[0201] For example, when the source artificial intelligence model (200) is determined to be in Graph execution mode (315a), the computing device (100) may determine model optimization tool(s) to be executed or applied to the source artificial intelligence model (200) within the range of a model optimization tool set including a compressor (375), a quantizer (380), and / or a graph optimizer (385) for the source artificial intelligence model (200). For example, when the source artificial intelligence model (200) is determined to be in Graph execution mode (315a), the computing device (100) may determine model optimization tool(s) to be executed or applied to the source artificial intelligence model (200) within the range of a model optimization tool set including a compressor (375), a quantizer (380), a graph optimizer (385), and / or a simulator (390) for the source artificial intelligence model (200). For example, when the source artificial intelligence model (200) is determined to be in Graph execution mode (315a), the computing device (100) can use an intermediate representation manager (330) to convert the source artificial intelligence model into an intermediate representation structure and allow model optimization tools (e.g., 375, 380, 385 and / or 390) to be executed on the intermediate representation structure.

[0202] In one embodiment, the intermediate representation manager (330) may be configured to generate a target artificial intelligence model (330) by converting an artificial intelligence model into an intermediate representation structure (370) and applying optimization and / or converting to the intermediate representation structure (370). For example, the intermediate representation manager (330) may operate as a component within a computing device (100). For example, the intermediate representation manager (330) may mean software that defines and / or performs the aforementioned functions within the computing device (100).

[0203] In one embodiment, the intermediate representation manager (330) may include a loader manager (340), a model editor (350), and / or an export manager (360).

[0204] In one embodiment, a loader manager (340) may be configured to convert an artificial intelligence model into an intermediate representation structure (370). The loader manager (340) may operate as a component within a computing device (100) and may refer to software that performs the function of converting into an intermediate representation structure (370). The loader manager (340) may convert the artificial intelligence model into a hierarchical structure composed of a node corresponding to an atomic operator and a module which is a set of nodes. The loader manager (340) may convert the artificial intelligence model into a graph structure composed of nodes, modules, and tensors. The loader manager (340) may generate an intermediate representation structure (370) that represents the dependency or hierarchical relationship between a node independent of the framework and a module dependent on the framework. For example, the loader manager (340) may generate a loader corresponding to the artificial intelligence model, and the method of generating an intermediate representation structure (370) for the artificial intelligence model through the loader may also be possible depending on the implementation mode. The loader manager (340) can be operated to take an artificial intelligence model (e.g., source artificial intelligence model) as input and an intermediate representation structure (370) as output. For example, the loader manager (340) can store and manage multiple loaders. For example, the loader manager (340) can manage loaders prepared for each framework of the artificial intelligence model, such as a TensorFlow loader, a TensorFlow Lite loader, an ONNX loader and / or a PyTorch loader. The loader manager (340) can determine a loader suitable for the artificial intelligence model to generate an intermediate representation structure (370) based on the parsing result of the artificial intelligence model, and call the determined loader to generate an intermediate representation structure (370) corresponding to the artificial intelligence model.In one embodiment, a loader or loader manager (340) can wrap the input of an artificial intelligence model in a Proxy and convert operators dependent on the framework of the artificial intelligence model into modules to create an intermediate representation structure in the form of a graph.

[0205] In one embodiment, the model editor (350) may be configured to modify or transform the generated intermediate representation structure (370). The model editor (350) may be configured to work with model optimization tools to generate an intermediate representation structure (370) that reflects optimization according to the model optimization tools. The model editor (350) may be configured to modify the intermediate representation structure (370) according to the results of channel axis transformation and / or dimension transformation. The model editor (350) may mean software that operates as a component within the computing device (100) and performs the function of changing or editing the intermediate representation structure (370).

[0206] In one embodiment, the export manager (360) may be configured to convert an intermediate representation structure (370) to correspond to the target framework of the target artificial intelligence model (300). The export manager (360) may be operated to take the intermediate representation structure (370) as input and the target artificial intelligence model (300) as output. The export manager (360) may be configured to combine the nodes of the intermediate representation structure (370) to convert (e.g., convert) them into module(s) dependent on the target framework. For example, the export manager (360) may create a converter that converts the model to the corresponding framework, and the model conversion to the target framework may be performed using the created converter. For example, the export manager (360) may store and manage multiple converters. For example, the export manager (360) may manage converters corresponding to the combination unit of the source framework and the target framework. For example, the export manager (360) can manage converters corresponding to the framework. For example, the export manager (360) can manage converters prepared for each framework of the artificial intelligence model, such as a TensorFlow converter, a TensorFlow Lite converter, an ONNX converter, and / or a PyTorch converter. The export manager (360) can determine a converter suitable for the target artificial intelligence model (300) to convert the intermediate representation structure (370) based on the target framework of the target artificial intelligence model (300), and call the determined converter to convert the intermediate representation structure (370) into the target artificial intelligence model (300).

[0207] In one embodiment, the export manager (360) may include one or more converters that convert an intermediate representation structure in the present disclosure into another form of intermediate representation structure (e.g., ONNX, PyTorch Fx, and TFlite, etc.). The converters may perform abstraction by representing a hierarchical graph of nodes and modules as a graph of modules. The converters may lower the level of abstraction by representing the graph of modules as a hierarchical relationship composed of nodes and modules. The converters may raise the level of abstraction by representing the hierarchical relationship composed of nodes and modules as a graph of modules. The export manager (360) may include one or more compilers that perform lowering or low-level conversion by converting an intermediate representation structure in the present disclosure into an intermediate representation structure of a low-level representation such as XLA HLO, LLVM, etc.

[0208] Levels of abstraction in the present disclosure may include levels of abstraction related to the model hierarchy and / or levels of abstraction related to the execution of the model.

[0209] In one embodiment, the level of abstraction associated with the model hierarchy may represent a level of abstraction within the model hierarchy, such as when a layer of the source AI model is concretized one step into nodes during the process of converting from the source AI model or other intermediate representation structure of the present disclosure to the intermediate representation structure of the present disclosure. For example, from the perspective of a converter, an abstraction associated with the model hierarchy or a conversion of the level of abstraction associated with the model hierarchy may be used. As an example of an abstraction associated with the model hierarchy, if a module corresponding to a layer of the model is converted into a structure including one or more nodes, the level of abstraction may be expressed as lowered.

[0210] In one embodiment, the level of abstraction related to the execution of a model may represent the level of abstraction used in a situation where relatively high-level source code is compiled (e.g., transformed) (or inversely transformed therefrom) into a relatively low-level form closer to machine code. For example, the higher the level of abstraction (high level), the closer it is to a conceptual representation that is easy for humans to understand, and the lower the level of abstraction (low level), the closer it is to a concrete representation that is easy for a machine to understand and execute. For example, from the perspective of a compiler, abstractions related to the execution of a model or transformations of the level of abstraction related to the execution of a model may be used. As an example of abstraction related to the execution of a model, when an expression written in the form of source code is compiled into an expression written in machine code, it may be expressed as having a lower level of abstraction.

[0211] In one embodiment, model optimization tools may include a trainer (320a), a compressor (320b and 375), a graph optimizer (385), a quantizer (320c and 380), and / or a simulator (390). Model optimization tools may include various types of trainers, various types of compressors, various types of graph optimizers, various types of quantizers, and / or various types of simulators. In one embodiment, different sets of model optimization tools may be used depending on the execution mode of the source artificial intelligence model (200). The trainer (320a), the compressor (320c), and the quantizer (320c) are model optimization tools that can be operated in Eager execution mode. The graph optimizer (385), the compressor (375), the quantizer (380), and / or the simulator (390) are model optimization tools that can be operated in Graph execution mode.

[0212] In one embodiment, the interconnection relationship between model optimization tools can be managed as a transformation scenario.

[0213] In one embodiment, the learner (320a) can generate the learning results of the artificial intelligence model.

[0214] In one embodiment, the compressor (320b and 375) can lighten the artificial intelligence model, such as through filter decomposition and / or pruning. The compressor can measure locations where the lightening technique is applicable and perform compression based on platform information using the measured locations. For example, the Advanced compressor (320b) can perform compression of the model through a compression technique operating in Eager execution mode (315a). For example, the compressor (375) can perform compression of the model through a compression technique operating in Graph execution mode (315b).

[0215] In one embodiment, the quantizers (320c and 380) can perform quantization of the artificial intelligence model based on data types supported by the hardware and / or compiler. For example, the Advanced quantizer (320c) can perform quantization of the model through a quantization technique operating in Eager execution mode (315a). For example, the quantizer (380) can perform quantization of the model through a quantization technique operating in Graph execution mode (315b).

[0216] In one embodiment, the graph optimizer (385) may remove operators from the model that are not required by the hardware and / or compiler, change unsupported operators into supported operators, change unsupported operators into a combination of supported operators, and / or change supported operators into operators for more efficient operation.

[0217] In one embodiment, the simulator (390) can generate an expected execution result when the artificial intelligence model is executed on a platform. The simulator (390) can generate an expected execution result of the artificial intelligence model by simulating a runtime running on hardware. The simulator (390) can generate an expected execution result of the artificial intelligence model by simulating a runtime running on hardware on an intermediate representation structure (370) without executing the actual model on hardware.

[0218] In one embodiment, model optimization tools (375, 380, 385, and 390) may be executed or applied on an intermediate representation structure. Accordingly, optimization, transformation, and simulation of the model may be possible without directly executing the actual model on an actual platform (e.g., hardware).

[0219] In one embodiment, the target artificial intelligence model (300) may refer to an artificial intelligence model that is the subject of transformation of the framework. For example, the target artificial intelligence model (300) may refer to an artificial intelligence model to which optimization and / or transformation has been applied. For example, the performance according to the expected simulation results of the target artificial intelligence model (300) may be performed on an intermediate representation structure (370) by a simulator (390). For example, the target artificial intelligence model (300) may be generated as the intermediate representation structure (370) is converted by a converter or export manager (360).

[0220] In one embodiment, the target artificial intelligence model (300) may include a static model and a dynamic model. For example, the target artificial intelligence model (300) may take the form of a static graph of a deep learning framework such as TensorFlow or Onnx. For example, the target artificial intelligence model (300) may take the form of a dynamic graph in the form of an Eager execution mode of a deep learning framework such as PyTorch. As an example, not a limitation, if it is determined that the target artificial intelligence model (300) will take the form of a dynamic graph (e.g., according to user input or a predetermined rule), the computing device (100) may apply a static to dynamic conversion when generating or outputting the target artificial intelligence model (300).

[0221] FIG. 4 is a flowchart exemplarily illustrating a method for converting a source artificial intelligence model into a target artificial intelligence model according to one embodiment of the present disclosure.

[0222] The exemplary flowchart in FIG. 4 can be performed by a computing device (100). Depending on the implementation mode, some steps in the example in FIG. 4 may be deleted and / or additional steps may be added to the example in FIG. 4.

[0223] In one embodiment, the computing device (100) can receive a source artificial intelligence model executable in a first framework (410).

[0224] In one embodiment, the source artificial intelligence model represents an artificial intelligence model input into a computing device (100). The source artificial intelligence model represents an artificial intelligence model that is the subject of conversion. The source artificial intelligence model may be received from a user terminal. The source artificial intelligence model may include identification information of the artificial intelligence model, a model file, framework information, and / or training dataset information. The source artificial intelligence model may be the subject of conversion into a target artificial intelligence model. The source artificial intelligence model may mean a model executable in a first framework (e.g., a source framework). The source artificial intelligence model may be the subject of conversion into a target artificial intelligence model operable in a second framework.

[0225] In one embodiment, the source artificial intelligence model may refer to a model abstracted to a first level of abstraction corresponding to a relatively high level. Such a source artificial intelligence model may be composed of components of the model's inference function unit.

[0226] In one embodiment, the computing device (100) can convert source operators included in the source artificial intelligence model into a source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework (420).

[0227] In one embodiment, the computing device (100) can convert a source artificial intelligence model into a source intermediate representation structure. For example, the computing device (100) can convert the source artificial intelligence model into an intermediate representation for converting it into an artificial intelligence model operable in a different framework. For example, the computing device (100) can convert the source artificial intelligence model into an intermediate representation for executing various model optimization tools. For example, the computing device (100) can convert the source artificial intelligence model into an intermediate representation for simulating the optimization or conversion results of the source artificial intelligence model. For example, the computing device (100) can generate a source intermediate representation structure that represents the internal structure of the source artificial intelligence model as a graph structure. For example, the source intermediate representation structure can be generated by a loader manager of the computing device (100). For example, the source intermediate representation structure can be generated by a loader corresponding to the source artificial intelligence model among a plurality of loaders.

[0228] In one embodiment, an intermediate representation structure corresponding to a source artificial intelligence model is referred to as a source intermediate representation structure. The source intermediate representation structure may be composed of source modules dependent on a first framework and nodes within the source modules. Each of the nodes here may correspond to an atomic operator independent of the framework. The expression "dependent of the framework" in this disclosure indicates that it varies according to a specific framework. Since each module within the intermediate representation structure is dependent on the framework, the form of the module and the information of the nodes within the module may be configured differently depending on the specific framework. For example, the source intermediate representation structure may be composed of source modules suitable for a first framework in which the source artificial intelligence model is executable. The source modules may be determined based on the first framework. Each of the nodes within the source modules may correspond to an atomic operator that is independent of the framework and cannot be further decomposed. For example, a computing device (100) may determine source modules corresponding to each of the source operators (e.g., source layers) constituting the source artificial intelligence model and determine the atomic operators within the source artificial intelligence model. Source modules can be generated by combining atomic operators to fit the first framework (i.e., source artificial intelligence model). The computing device (100) can generate a source intermediate representation structure that represents the source artificial intelligence model as source modules and nodes by converting each of the source operators (e.g., source layers) constituting the source artificial intelligence model into a source module composed of the nodes independent of the framework. The source modules can correspond to source operators (layers included in the source artificial intelligence model) that are dependent on the first framework.

[0229] In one embodiment, the source intermediate representation structure may include nodes, tensors connecting the nodes, source modules composed of one or more nodes representing information dependent on the first framework, and tensors connecting the source modules. The source intermediate representation structure may further include arguments representing information (or values) required to execute specific operators of the source artificial intelligence model. The arguments may include information for the execution of modules and / or nodes within the intermediate representation structure. Additionally, tensors in the source intermediate representation structure may be used as said arguments. For example, each of the modules and nodes of the intermediate representation structure may include arguments for their respective execution. Platform information and / or quantization information may be stored in tensors within the intermediate representation structure.

[0230] In one embodiment, the source module in the source intermediate representation structure may include an input tensor, an output tensor, a source operator dependent on the first framework, nodes existing within the source module, an original module at the time of creation of the source intermediate representation structure, and original nodes at the time of creation of the source intermediate representation structure. Here, the original module may refer to a module transformed from an operator or set of operators (or layers) of the source artificial intelligence model when creating the source intermediate representation structure. For example, the original module may be changed into an optimized or transformed form as model optimization tool(s) are executed. The node in the source intermediate representation structure may include an input tensor, an output tensor, an atomic operator corresponding to the node, and original nodes at the time of creation of the source intermediate representation structure. Here, the original node may refer to a node transformed from an atomic operator of the source artificial intelligence model when creating the source intermediate representation structure. For example, the original node may be changed into an optimized or transformed form as model optimization tool(s) are executed.

[0231] In one embodiment, the computing device (100) can generate a target intermediate representation structure including target modules dependent on the second framework and nodes independent of the framework by using information about the second framework to which the source artificial intelligence model is to be converted (430).

[0232] In one embodiment, the computing device (100) may change source modules into target modules dependent on the second framework. The target modules may be determined dependently on the second framework in which the target artificial intelligence model is executable. Like source modules, target modules may vary according to the framework. In order for the intermediate representation structure to be changed to be suitable for the first framework, the module(s) on the intermediate representation structure may be changed into first target module(s) corresponding to the first framework. In order for the intermediate representation structure created to be suitable for the first framework to be changed to be suitable for the second framework, the source module(s) corresponding to the first framework on the intermediate representation structure may be changed into second target module(s) corresponding to the second framework. The number and types of nodes included in the target modules may also vary according to the framework. In one embodiment, since the nodes included in the source intermediate representation structure are independent of the framework, they may be maintained during the process of converting to the target intermediate representation structure. However, as the model optimization tool(s) are executed on the intermediate representation structure, the nodes may be changed.

[0233] In one embodiment, the source intermediate representation structure may be generated by applying a second level of abstraction, which corresponds to a relatively low level, to the source artificial intelligence model. The source intermediate representation structure may be represented by a module of the first level of abstraction and a node of the second level of abstraction. The source intermediate representation structure may be generated by applying a low-level abstraction of the source artificial intelligence model. The source intermediate representation structure may refer to a structure to which the second level of abstraction is applied.

[0234] In one embodiment, the target intermediate representation structure may also be represented as a module of the first level of abstraction and a node of the second level of abstraction so as to correspond to the source intermediate representation structure. The target intermediate representation structure may refer to a structure to which the abstraction of the second level of abstraction (e.g., an abstraction concretized down to the node level) is applied.

[0235] In one embodiment, the target intermediate representation structure may include a target module composed of one or more nodes, representing information dependent on a second framework and a node corresponding to an atomic operator independent of the framework. The target intermediate representation structure may further include a tensor connecting the target modules and / or nodes. The target intermediate representation structure may further include an argument representing a value used for the operator to operate. Depending on the implementation mode, such an argument may be included within the range of the tensor.

[0236] In one embodiment, the computing device (100) can convert a source intermediate representation structure into a universal intermediate representation structure for conversion to a target framework different from the source framework. For example, the universal intermediate representation structure may refer to an intermediate representation structure that serves as a standard for conversion to various forms of frameworks. For example, a source intermediate representation structure corresponding to a source framework and a target intermediate representation structure suitable for a target framework may have a structure dependent on a specific framework. For example, a source intermediate representation structure corresponding to a source framework and a target intermediate representation structure suitable for a target framework may have a module structure dependent on the framework (e.g., source module(s) and / or target module(s)). For example, the universal intermediate representation structure may refer to an intermediate representation structure that is not dependent on a framework. For example, the universal intermediate representation structure may refer to a structure in which module(s) on the source intermediate representation structure have been removed. For example, the computing device (100) may generate a universal intermediate representation structure by modifying the source intermediate representation structure so that a single node is included in a single module. For example, a universal intermediate representation structure can be created by removing framework-dependent module(s) from the created intermediate representation structure. For example, a universal intermediate representation structure can be created as the intermediate representation structure is modified so that modules and nodes have a one-to-one inclusion relationship. In one embodiment, the computing device (100) can create a universal intermediate representation structure by removing source modules dependent on the first framework and retaining nodes independent of the framework from the source intermediate representation structure. In one embodiment, the computing device (100) can create a universal intermediate representation structure by modifying the source intermediate representation structure so that a single node is included in a single source model from the source intermediate representation structure.For example, a general intermediate representation structure may be composed of nodes of a second level of abstraction corresponding to a relatively low level. For example, a general intermediate representation structure composed of components of the second level of abstraction may be created as components of the first level of abstraction are removed from a source intermediate representation structure. For example, a general intermediate representation structure may be created as high-level abstraction removal is applied to the source intermediate representation structure.

[0237] In one embodiment, to perform a conversion between frameworks of an artificial intelligence model, a source intermediate representation structure to be converted is converted into a general-purpose intermediate representation structure, and a target intermediate representation structure, which is the result of the conversion, can be generated using the general-purpose intermediate representation structure. Since the technique according to one embodiment of the present disclosure uses an intermediate representation structure that can be universally applied to frameworks for conversion between frameworks, the technical effect of achieving resource-efficient conversion between various frameworks on an intermediate representation can be achieved.

[0238] In one embodiment, the computing device (100) can convert a general intermediate representation structure into a target intermediate representation structure that includes target modules dependent on the second framework and nodes independent of the framework, by using information about a second framework to which the source artificial intelligence model is to be converted. In one embodiment, the general intermediate representation structure used in the conversion process to the target intermediate representation structure may mean an intermediate representation independent of the framework. The general intermediate representation structure may include a combination of nodes corresponding to a minimum unit atomic operator that is universally applicable to a plurality of frameworks. The general intermediate representation structure may mean an intermediate representation composed only of nodes as modules are removed. The general intermediate representation structure may have a form in which modules include node(s) so as not to be dependent on the framework.

[0239] In one embodiment, the computing device (100) can convert a general-purpose intermediate representation structure into a target intermediate representation structure that includes target modules dependent on the second framework and nodes independent of the framework. For example, the computing device (100) can group nodes included in the general-purpose (or source) intermediate representation structure by using information about the second framework to which the source artificial intelligence model is to be converted as a grouping condition. For example, nodes on the general-purpose (or source) intermediate representation structure can be grouped using a table in which information about the second framework and information about combinations of nodes are mapped. As the nodes are grouped, target modules can be created. For example, a single target module may include one or more nodes. As a result of this grouping, target modules dependent on the second framework may be created. The target modules may correspond to target operators within the artificial intelligence model corresponding to the second framework.

[0240] In an additional embodiment, the computing device (100) may use a method of directly converting a source intermediate representation structure into a target intermediate representation structure without utilizing a general-purpose intermediate representation structure. For example, a target intermediate representation structure may be generated as source module(s) in the source intermediate representation structure are changed into target module(s) by using a target framework. For example, a target intermediate representation structure may be generated as source module(s) are changed into target module(s) by using a target framework and a mapping table that defines the correlation between modules and operators (or layers).

[0241] In one embodiment, a mapping table may be utilized in the process of generating a source intermediate representation structure, the process of generating an intermediate representation structure, and / or the process of generating a target intermediate representation structure. A technique according to one embodiment of the present disclosure may generate, construct, modify, manage, and / or utilize a mapping table for mutual conversion between various deep learning frameworks of an artificial intelligence model and intermediate representation structures defined in the present disclosure. The mapping table may define a mapping relationship between a framework-specific operator and an atomic operator (e.g., a node) of the intermediate representation structure of the present disclosure. For example, such a mapping table may be included in an intermediate representation manager (230). For example, the mapping table may be utilized in the process of generating and converting intermediate representation structures in components such as a loader manager and / or an export manager.

[0242] In one embodiment, the target module in the target intermediate representation structure may include: an input tensor, an output tensor, a target operator dependent on the second framework, nodes existing within the target module, the original module at the time of creation of the target intermediate representation structure, and the original nodes at the time of creation of the source intermediate representation structure. The nodes in the target intermediate representation structure may include an input tensor, an output tensor, an atomic operator corresponding to the node, and the original nodes at the time of creation of the source intermediate representation structure.

[0243] In one embodiment, the computing device (100) can define correlations between operators and nodes (or modules) within an intermediate representation structure using a mapping table. Depending on the implementation mode, such a mapping table may be used interchangeably with a lookup table.

[0244] For example, the computing device (100) may use a lookup table when generating a source intermediate representation structure from a source artificial intelligence model. For example, the computing device (100) may convert source operators into a combination of nodes within the source intermediate representation structure. For example, the computing device (100) may convert source operators included in the source artificial intelligence model into a combination of nodes by using a lookup table that represents the conversion relationship between source operators and nodes of the first framework. For example, a source intermediate representation structure may be generated by combining nodes that have a mapping relationship with each of the source operators of the first framework. For example, as a first lookup table corresponding to the first framework is used, a source intermediate representation structure including nodes determined based on information stored in the first lookup table may be generated.

[0245] In one embodiment, when generating a source intermediate representation structure, the computing device (100) may use a lookup table that includes a transformation relationship between source operators and nodes of a first framework and an argument used for the operator to operate in the first framework. For example, the computing device (100) may use a lookup table that includes transformation relationships and arguments to transform each source operator included in the source artificial intelligence model into a source module which is a combination of nodes, and to transform the arguments of the source operators into arguments of the nodes included in the source module.

[0246] In one embodiment, a lookup table may be used to identify combinations of nodes that match operators of a specific framework. In one embodiment, the lookup table may be configured to match combinations of frameworks and operators with nodes. In one embodiment, the lookup table may be configured to match combinations of frameworks and operators with combinations of nodes. For example, the computing device (100) can identify combinations of nodes that match combinations of source operators of a source artificial intelligence model and a first framework by using a lookup table that identifies combinations of nodes that match combinations of types of frameworks and types of operators. The computing device (100) can convert each source operator constituting the source artificial intelligence model into a source module by configuring the identified combinations of nodes into a single module. Specific examples of conversion into source modules will be described later in the drawings below.

[0247] In one embodiment, the computing device (100) may use a lookup table when generating a target intermediate representation structure. The lookup table may represent a transformation relationship between nodes corresponding to the target operators and atomic operators of the second framework. The computing device (100) may use the lookup table to determine a method of recombination of nodes for compatibility with the second framework. In an additional embodiment, among a plurality of types of lookup tables, the computing device (100) may select a lookup table related to the second framework and use the selected lookup table to determine a method of recombination of nodes for compatibility with the second framework. In one embodiment, the computing device (100) can generate target modules dependent on a second framework by grouping nodes using a determined recombination method. Recombination of nodes in the present disclosure may mean rearranging nodes within a source intermediate representation structure to correspond to a target intermediate representation structure. Recombination of nodes in the present disclosure may mean adaptively rearranging nodes within a source intermediate representation structure according to a target intermediate representation structure. Recombination of nodes in the present disclosure may mean adjusting the arrangement positions, sequence relationships, and / or arrangement relationships of nodes within an intermediate representation structure.

[0248] In one embodiment, the lookup table may have a data structure including a key and a value. In one embodiment, when the computing device (100) converts source operators of a source artificial intelligence model into a source intermediate representation structure, it may use a lookup table that uses a framework-specific operator as a key and a node corresponding to an atomic operator as a value. By using the lookup table, source operators included in the source artificial intelligence model may be converted into a combination of nodes. In an additional embodiment, when the computing device (100) generates a target intermediate representation structure from a source intermediate representation structure or from a universal intermediate representation structure, it may use a lookup table that uses a framework-specific operator as a key and a node corresponding to an atomic operator as a value. By using the lookup table, a combination of nodes of the source intermediate representation structure or the universal intermediate representation structure may be converted into a combination of nodes of the target intermediate representation structure.

[0249] In one embodiment, the computing device (100) may generate a mapping table (or lookup table) by mapping each of the model operators to each of the modules within the intermediate representation structure. For example, the computing device (100) may generate a mapping table by using an execution-based analysis method based on the execution of a test artificial intelligence model and / or an embedding-based analysis method based on embedding transformation between an atomic operator independent of the model operator and the framework. For example, the computing device (100) may generate a mapping table by determining a module corresponding to a model operator using an execution-based analysis method and / or an embedding-based analysis method, and by mapping the determined module to the model operator. For example, the mapping table may include version information of the framework, identification information of the model operator within the mapping table, input / output information of the model operator, identification information of one or more nodes included in the module mapped to the model operator, input / output information of one or more nodes, and / or the execution order of each node. For example, the mapping table may define a mapping relationship between an operator (or layer) of an artificial intelligence model and a node or set of nodes of the intermediate representation structure. The computing device (100) can convert from a source artificial intelligence model to a source intermediate representation structure using a mapping table. The computing device (100) can convert from a target intermediate representation structure to a target artificial intelligence model using a mapping table.

[0250] In one embodiment, the computing device (100) may generate a reverse mapping table that defines a mapping relationship between an atomic operator and a model operator by using a mapping relationship in a forward mapping table that defines a mapping relationship between a model operator and an atomic operator. For example, the forward mapping table may be used to generate a source intermediate representation structure from a source artificial intelligence model. For example, the reverse mapping table may be used to generate a target artificial intelligence model from a target intermediate representation structure.

[0251] In one embodiment, the computing device (100) can determine a module corresponding to a model operator based on the type of the framework. In one embodiment, the computing device (100) can determine a module corresponding to a model operator in a different way depending on the type of the framework. In one embodiment, the computing device (100) can vary the method of determining a module corresponding to a model operator dependently on the type of the framework.

[0252] In one embodiment, the computing device (100) can determine a module corresponding to a model operator based on a first execution-based analysis method that traces atomic operators called during the execution of a test artificial intelligence model, provided that the type of the framework is a framework that supports a function of converting into an intermediate representation structure (e.g., a Torch-based framework). For example, the computing device (100) can generate a first test artificial intelligence model including model operators and convert the model operators included in the first test artificial intelligence model into a first intermediate representation structure expressed in units of atomic operators by executing the first test artificial intelligence model. Then, based on the model operator information, the computing device (100) can identify the combination and execution order of atomic operators (e.g., nodes) corresponding to each of the model operators within the first intermediate representation structure. Based on the identified combination and execution order of atomic operators, the computing device (100) can determine the module corresponding to each of the model operators. In this way, model operators and corresponding modules can be mapped to generate a mapping table. For example, a combination of atomic operators (e.g., nodes) corresponding to a model operator can be determined as a module.

[0253] In one embodiment, the computing device (100) may determine a module corresponding to a model operator based on a second execution-based analysis method or an embedding-based analysis method that compares the execution results of a test artificial intelligence model, when the type of the framework is a framework that does not support a function of converting to an intermediate representation structure (e.g., a non-torch-based framework). For example, the second execution-based analysis method may include determining a module mapped to a model operator(s) of a specific artificial intelligence model by comparing the execution results of a specific input data of a specific artificial intelligence model (e.g., a second test artificial intelligence model) with the execution results of candidate modules for said specific input data.

[0254] In one embodiment, the computing device (100) may determine a module corresponding to a model operator after generating a mapping table based on a first execution-based analysis method that tracks atomic operators called during the execution of a test artificial intelligence model, when the type of the framework is a framework that does not support a function of converting to an intermediate representation structure. For example, when the type of the framework is a framework that does not support a function of converting to an intermediate representation structure, the computing device (100) may generate a candidate mapping table corresponding to the framework based on a second execution-based analysis method using a threshold error or an embedding-based analysis method using a threshold embedding similarity. The candidate mapping table may correspond to a table in which one or more candidate modules are mapped to one target model operator. The candidate mapping table may be generated using modules predetermined for a framework in which the type of the framework supports a function of converting to an intermediate representation structure. The computing device (100) may determine a module corresponding to each of the model operators by comparing the execution result of each of the model operators with the execution result of each of the one or more candidate modules mapped within the candidate mapping table.

[0255] In one embodiment, the second execution-based analysis method may be implemented with the following operations. For example, a computing device (100) may generate a second test artificial intelligence model including model operators that are subject to mapping. Based on the execution result of the first input data of the second test artificial intelligence model and the execution result of a plurality of pre-stored candidate modules on the first input data, the computing device (100) may determine a module corresponding to each of the model operators included in the second test artificial intelligence model. A mapping relationship between the model operators and the determined modules may be established. For example, a plurality of pre-stored candidate modules may correspond to pre-determined modules based on the first execution-based analysis method.

[0256] In one embodiment, when a second test artificial intelligence model is composed of a single first model operator, the computing device (100) can determine a module corresponding to the first model operator included in the second test artificial intelligence model by calculating an error between the execution result of the first input data of the first model operator and the execution result of the first input data of the plurality of pre-stored candidate modules, and using the calculated error to determine the module corresponding to the first model operator. For example, the computing device (100) can determine a module corresponding to the first model operator among the one or more candidate modules identified among the plurality of pre-stored candidate modules, wherein the error is less than or equal to a threshold error, and by comparing the execution result of the second input data of the first model operator with the execution result of the second input data of the identified one or more candidate modules.

[0257] In one embodiment, when a second test artificial intelligence model is composed of a plurality of model operators, the computing device (100) can calculate an error between the execution result of the first input data of the second test artificial intelligence model and the execution result of the first input data of candidate module combinations composed of the plurality of pre-stored candidate modules, and can determine a module corresponding to each of the model operators using the calculated error. For example, the computing device (100) can identify one or more candidate module combinations among the candidate module combinations in which the error is less than or equal to a threshold error, and can determine a module corresponding to each of the model operators within the identified one or more candidate module combinations by comparing the execution result of the third input data of each of the model operators included in the second test artificial intelligence model with the execution result of the third input data of each of the candidate modules included in the identified one or more candidate module combinations.

[0258] In one embodiment, the computing device (100) can compare an intermediate execution result (e.g., intermediate input / output value) for each model operator with respect to input data with the execution result of each candidate module for said input data in establishing a mapping relationship between a model operator and a module within an artificial intelligence model. Based on this comparison, a candidate module that is below a threshold error can be identified. A mapping relationship between the identified candidate module and the model operator can be established.

[0259] In one embodiment, the computing device (100) may use an embedding analysis method when determining a module corresponding to a model operator. This embedding analysis method may be implemented by generating a model operator embedding vector corresponding to a model operator and generating an atomic operator embedding vector corresponding to an atomic operator. As the vector similarity between the model operator embedding vector and the atomic operator embedding vector is calculated, the module corresponding to the model operator may be determined. For example, based on the vector similarity, a combination of atomic operators (e.g., nodes) corresponding to the model operator may be determined, and the combination of atomic operators may be configured to constitute a module. For example, the model operator embedding vector may be generated by performing a filtering process on a plurality of model operator tokens, in which model operator information corresponding to each model operator is tokenized, by deleting tokens corresponding to predefined stopwords or selecting tokens corresponding to predefined keywords. Model operator information may include identification information, description information, input / output information, and implementation code for each model operator. The description information may represent information in which the description of each model operator is written in natural language text. The input / output information may include data type, shape, and order information of the input and output values ​​of each model operator. For example, an atomic operator embedding vector may be generated by performing a filtering process on multiple atomic operator tokens, in which information regarding the atomic operators is tokenized, by deleting tokens corresponding to predefined stop words or selecting tokens corresponding to predefined core keywords.

[0260] In one embodiment, the computing device (100) can generate a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into target operators by converting the target modules of the target intermediate representation structure into target operators executable in a second framework (440).

[0261] In one embodiment, the target AI model may refer to the result of the transformation (or converting) of the source AI model. In one embodiment, the target AI model may be generated as a result of converting the target intermediate representation structure. In one embodiment, the target AI model may be converted in such a way that the modules of the target intermediate representation structure are converted into target operators. In one embodiment, the target AI model may refer to an AI model executable on the second framework. In one embodiment, a mapping table (e.g., a lookup table) representing the correlation between target operators and nodes may be used during the conversion process of the target AI model. As the mapping table is used, target modules on the target intermediate representation structure may be converted into target operators. As this mapping table is used, nodes or combinations of nodes on the target intermediate representation structure may be converted into target operators. The target AI model converted from the target intermediate representation structure may be provided to a user terminal. As the target AI model is executed, inference of the AI ​​model in which the framework has been converted from the first framework to the second framework may be performed.

[0262] In one embodiment, the computing device (100) determines a converter corresponding to a target artificial intelligence model among a plurality of converters and can convert an intermediate representation structure into a target artificial intelligence model including target speakers using the determined converter. In one embodiment, the computing device (100) determines a converter corresponding to a second framework among a plurality of converters and can convert an intermediate representation structure into a target artificial intelligence model including target speakers using the determined converter.

[0263] In one embodiment, the computing device (100) can convert a target intermediate representation structure into a target artificial intelligence model including target operators using a lookup table containing various information. For example, the computing device (100) can perform conversion between combinations of nodes and target operators using mapping relationships stored in the lookup table. For example, the computing device (100) can convert arguments stored in the lookup table into arguments of target operators. For example, the computing device (100) can generate a target artificial intelligence model using information stored in the tensor of the lookup table.

[0264] In one embodiment, the computing device (100) can convert a target intermediate representation structure into a target artificial intelligence model using a lookup table that defines a mapping relationship between a second framework and a combination of nodes. In one embodiment, the computing device (100) can generate a target artificial intelligence model corresponding to the second framework by converting arguments within the intermediate representation structure into arguments of target operators using a lookup table. In one embodiment, the computing device (100) can convert the arguments of nodes included in the target modules into arguments of target operators of the second framework by using a lookup table that indicates a conversion relationship between the second framework and nodes and arguments used for an operator to operate in the second framework. Through this conversion, a target artificial intelligence model compatible with the second framework can be generated.

[0265] In one embodiment, the computing device (100) can convert the target intermediate representation structure into a target artificial intelligence model by using information contained in a tensor within the target intermediate representation structure. In one embodiment, the computing device (100) can generate a quantization operator and a dequantization operator corresponding to a second framework by using quantization information of a tensor contained in the target intermediate representation structure. When converting the target modules of the target intermediate representation structure into target operators executable in the second framework, the quantization operator and the dequantization operator may be added to the target operators. The quantization operator and the dequantization operator may be placed before and after the target operator, respectively, or the dequantization operator and the quantization operator may be placed before and after the target operator, respectively.

[0266] In one embodiment, the computing device (100) may identify the execution mode of a source artificial intelligence model and vary the method of converting to a source intermediate representation structure according to the identified execution mode. For example, if the received source artificial intelligence model is identified as a first mode in which the model is executed after representing operations as a graph, the computing device (100) may convert the received source artificial intelligence model into a source intermediate representation structure. For example, if the received source artificial intelligence model is identified as a second mode in which the model is executed in units of operations without representing operations as a graph, the computing device (100) may apply a model optimization tool on the source artificial intelligence model and perform conversion from the source artificial intelligence model to which the model optimization tool is applied into a source intermediate representation structure.

[0267] In one embodiment, after model optimization tools are applied or executed on a target intermediate representation structure, a conversion operation to a target artificial intelligence model may be performed. For example, a computing device (100) may modify target modules included in a target intermediate representation structure using a model optimization tool comprising at least one of a compressor, a graph optimizer, and a quantizer. For example, the computing device (100) may modify the configuration of target modules included in a target intermediate representation structure using a model optimization tool comprising at least one of a compressor, a graph optimizer, and a quantizer. Then, the computing device (100) may generate the target artificial intelligence model executable in a second framework by converting the target modules with modified configurations of the target intermediate representation structure into target operators executable in a second framework.

[0268] In one embodiment, the computing device (100) may use platform information during the process of converting to a target artificial intelligence model. For example, the computing device (100) may use or consider hardware information and / or compiler information during the process of converting to a target artificial intelligence model. Depending on the platform information, the method of conversion and / or the result of conversion may vary.

[0269] In one embodiment, when the target artificial intelligence model receives information about the target hardware to be deployed, the information about the target hardware may be stored in the target module and tensor in the target intermediate representation structure. The computing device (100) may perform conversion to the target artificial intelligence model by selecting an operator that is supported by the target hardware using the information about the target hardware. The computing device (100) may perform conversion to the target artificial intelligence model by removing an operator that is not supported by the target hardware using the information about the target hardware. For example, the computing device (100) may create the target artificial intelligence model by removing a target module or node corresponding to an operator that is not supported by the target hardware from the target intermediate representation structure using the information about the target hardware. For example, the information about the target hardware may be determined on the target intermediate representation structure or received from a user terminal.

[0270] In one embodiment, the computing device (100) may perform conversion to a target artificial intelligence model by utilizing or considering information about the target hardware and the target compiler together. In one embodiment, when the target artificial intelligence model receives information about the target hardware and the target compiler to be deployed, a node supported by the target hardware and the target compiler and a module including said supported node may be added to the target intermediate representation structure. In one embodiment, when the target artificial intelligence model receives information about the target hardware and the target compiler to be deployed, a target operator supported by said target hardware and the target compiler may be generated during the conversion process to the target artificial intelligence model. In one embodiment, when the target artificial intelligence model receives information about the target hardware and the target compiler to be deployed, a node not supported by said target hardware and the target compiler and a module including said unsupported node may be deleted from the target intermediate representation structure. In one embodiment, when a target artificial intelligence model receives information about the target hardware and / or target compiler to be deployed, a node that is not supported by the target hardware and / or target compiler and a module containing the unsupported node on the target intermediate representation structure may be changed to a node and module that are supported by the target hardware and / or target compiler. In one embodiment, when a target artificial intelligence model receives information about the target hardware and / or target compiler to be deployed, conversion to a target artificial intelligence model may be performed in such a way that a node that is not supported by the target hardware and / or target compiler and a module containing the unsupported node on the target intermediate representation structure are changed to a target operator that is supported by the target hardware and / or target compiler.

[0271] In one embodiment, the computing device (100) may perform an optimization operation or a performance transformation operation on a target intermediate representation structure using platform information. For example, the computing device (100) may determine a graph optimization method in the target intermediate representation structure based on the platform information. There may be a correlation between the platform information and the graph optimization method. For example, when the computing device (100) receives platform information (e.g., information on the target hardware) on which the target artificial intelligence model is to be deployed, it may delete a node or module on the target intermediate representation structure using the platform information. For example, when the computing device (100) receives platform information (e.g., information on the target hardware) on which the target artificial intelligence model is to be deployed, it may delete at least one node among the nodes included in the target intermediate representation structure using the platform information. For example, when the computing device (100) receives platform information (e.g., information of target hardware) to which the target artificial intelligence model is to be deployed, it can use the platform information to move at least one node among the nodes included in the target intermediate representation structure from the first target module to the second target module on the target intermediate representation structure.

[0272] In one embodiment, the computing device (100) can generate a target artificial intelligence model by abstracting a target intermediate representation structure to a high level. A target artificial intelligence model can be generated by increasing the level of abstraction of a target intermediate representation structure expressed at a second level of abstraction. For example, a target artificial intelligence model can be generated by removing components corresponding to the second level of abstraction from the target intermediate representation structure. For example, the target artificial intelligence model can be expressed at a first level of abstraction. For example, a target artificial intelligence model can be generated by abstracting the target intermediate representation structure to a high level, and the target artificial intelligence model can be expressed at a first level of abstraction by this abstraction.

[0273] FIG. 5 illustrates an intermediate representation structure (520) according to one embodiment of the present disclosure.

[0274] In one embodiment, the artificial intelligence model (510) may include operators (510a and 510b). The operators (510a and 510b) may represent units that perform specific functions within the artificial intelligence model. The operators (510a and 510b) of the artificial intelligence model (510) may be used interchangeably with layers.

[0275] In one embodiment, the artificial intelligence model (510) may correspond to a source artificial intelligence model to which conversion to a target artificial intelligence model is performed using an intermediate representation structure (520). In one embodiment, the artificial intelligence model (510) may correspond to an artificial intelligence model to which model optimization tools are executed or applied on the intermediate representation structure (520). In one embodiment, the artificial intelligence model (510) may correspond to an artificial intelligence model to which simulation is performed on the intermediate representation structure (520).

[0276] In one embodiment, the intermediate representation structure (520) may include a source intermediate representation structure corresponding to a source artificial intelligence model, a target intermediate representation structure corresponding to a target artificial intelligence model, and / or a general-purpose intermediate representation structure.

[0277] In one embodiment, the intermediate representation structure (520) may have a graph structure as illustrated in FIG. 5. Nodes (530a, 530b, 530c, 530d, and 530e) and edges connecting the nodes (530a, 530b, 530c, 530d, and 530e) and modules (520a, 520b, and 520c) and edges connecting the modules (520a, 520b, and 520c) may form a graph structure. The intermediate representation structure (520) may include nodes (530a, 530b, 530c, 530d, and 530e) corresponding to atomic operators. Modules (520a, 520b, and 520c) may be formed as one or more nodes are combined. The intermediate representation structure (520) may include tensors (Tensor#2, Tensor#4, Tensor#6, Tensor#8, Tensor#M-6, Tensor#M-3 and Tensor#M). For example, the tensors may represent connections between nodes and / or connections between modules, and may represent the inputs and outputs of nodes and the inputs and outputs of modules.

[0278] In one embodiment, a first operator (510a) in an artificial intelligence model (510) can be converted into a first module (520a) on an intermediate representation structure (520). In this conversion process, a set of nodes (530a, 530b, and 530c) corresponding to the first operator (510a) can be determined. This set of nodes (530a, 530b, and 530c) can be determined, for example, using a mapping table (or lookup table). One or more atomic operators representing the first operator (510a) are determined, and these atomic operators can be represented as a set of nodes (530a, 530b, and 530c). When the first module (520a) is configured, tensors (Tensor #2, Tensor #4, Tensor #6, Tensor #8) associated with the first module (520a) may be included. Using the same method, the second operator (510b) in the artificial intelligence model (510) may be converted to another module (e.g., Module #M-2 (520c)) on the intermediate representation structure (520).

[0279] In one embodiment, a node in an intermediate representation structure (520) may include: identification information of the node (ID and / or name), information identifying the input tensor of the node, information identifying the output tensor of the node, identification information of an atomic operator independent of the framework, information required to implement the atomic operator (e.g., an argument), information regarding the tensor, information regarding the argument, and / or node information at the time of the initial creation of the source intermediate representation structure.

[0280] In one embodiment, a module in an intermediate representation structure (520) may include: identification information (ID and / or name) of the module, information about nodes existing within the module, information about executable operators dependent on the framework, information required to execute said operators, information about tensors, information about arguments, input tensors of the module, output tensors of the module, information about the original module, and information about the original node within the module. For example, if the intermediate representation structure (520) is a source intermediate representation structure, the original module may include module information at the time of the initial creation of the source intermediate representation structure. For example, if the intermediate representation structure (520) is a target intermediate representation structure, the original module may include module information at the time of the initial creation of the target intermediate representation structure. For example, information about the original node may include node information at the time of the initial creation of the source intermediate representation structure. This information about the original node is not changed even after the source representation structure is converted into a target intermediate representation structure.

[0281] In one embodiment, the tensor in the intermediate representation structure (520) may include: identification information of the tensor (ID and / or name), identification information of the node outputting the tensor, identification information of the node using the tensor as input, the value of the tensor (e.g., a predefined value such as a weight), information indicating whether the parameter is in a learnable form, external storage location information, shape information of the tensor, size information of the tensor, information on whether quantization is in progress and / or information on quantization-related parameters. For example, the activation value may be defined as None in the value of the tensor.

[0282] While there are various deep learning frameworks that support the efficient development, training, and / or deployment of artificial intelligence models, each of these frameworks possesses unique advantages and disadvantages depending on its purpose. Certain frameworks are favored by many researchers because they have a relatively very low barrier to entry for AI model development, but they may have disadvantages in terms of optimization, which can lead to reduced performance and efficiency for execution on various devices. To compensate for the advantages and disadvantages of each framework or to use a runtime optimized for a specific device, conversion between deep learning frameworks is essential. For example, ONNX supports a fixed graph, and conversion to other frameworks is not readily available. Through a novel intermediate representation structure according to one embodiment of the present disclosure, a technical effect can be derived in that conversion using computing resources between various frameworks can be achieved. By utilizing the intermediate representation structure (520) of the present disclosure, the user can efficiently derive an AI model optimized for various frameworks and platforms using the same model.

[0283] According to one embodiment of the present disclosure, when generating a graph structure (intermediate representation structure (520)) for an artificial intelligence model, for a framework that allows dynamic control flow, a proxy for input data is generated to enable response to dynamic control flow and Eager execution mode. Additionally, according to one embodiment of the present disclosure, for an artificial intelligence model in a framework that does not allow dynamic control flow, a technical effect can be achieved in that mutual compatibility is made possible by fragmenting the model.

[0284] Through the intermediate representation structure (520) in the present disclosure, a technical effect can be achieved in that both static models and dynamic models can be received as input. Through the intermediate representation structure (520) in the present disclosure, a technical effect can be achieved in that models optimized by passing through model optimization tools can also be received as input. Quantization-related information can be stored in a tensor-type data structure including activation and / or weights within the intermediate representation structure (520), thereby achieving a technical effect in that the quantization information of the model can be stored regardless of the framework.

[0285] An intermediate representation structure (520) according to one embodiment of the present disclosure may allow artificial intelligence models of various frameworks, which are optimized in a framework-dependent manner, to be efficiently converted into artificial intelligence models of a specific framework. Since the intermediate representation structure (520) according to one embodiment of the present disclosure is configured to include a minimum unit of operator called a node independent of the framework, it may maintain compatibility with various frameworks regardless of operators (layers) defined differently between frameworks. Since the nodes in the intermediate representation structure (520) are grouped into modules corresponding to model operators of various frameworks, efficient conversion to an operator (layer) of a specific framework may be allowed. Through this, conversion and / or compilation to a specific framework may be performed from an intermediate representation structure (520) in the form of a graph composed of nodes, regardless of the input or output framework. In one embodiment, a computing device (100) may obtain an artificial intelligence model (510) (e.g., a source artificial intelligence model) in which a first level of abstraction has been achieved. In one embodiment, the computing device (100) may abstract the artificial intelligence model (510) to a first level. Through the first level of abstraction, the artificial intelligence model (510) may be configured to include components (510a and 510b) of inference function units. For example, the artificial intelligence model (510) may be represented by one or more modules (510a and 510b) through the first level (e.g., high level) of abstraction.

[0286] In one embodiment, the computing device (100) can abstract an artificial intelligence model (510) at a first level of abstraction to a second level of abstraction. For example, the computing device (100) can generate an intermediate representation structure (520) (e.g., a source intermediate representation structure) by configuring the artificial intelligence model (510) at the first level of abstraction through a lower level of abstraction unit. The computing device (100) can generate an intermediate representation structure (520) by abstracting the artificial intelligence model using a lower level of abstraction unit. The computing device (100) can generate an intermediate representation structure (520) by abstracting the artificial intelligence model (510) to a second level of abstraction. An intermediate representation structure (520) may be created to include modules (520a, 520b, and 520c) corresponding to a first level of abstraction and nodes (530a, 530b, 530c, 530d, and 530e) corresponding to a second level of abstraction. A computing device (100) may create an intermediate representation structure (520) from an artificial intelligence model (510) by lowering the level of abstraction. A computing device (100) may create an intermediate representation structure (520) through hierarchical abstraction of the first level of abstraction and the second level of abstraction.

[0287] In one embodiment, the computing device (100) can generate an intermediate representation structure (e.g., a general-purpose intermediate representation structure) composed of nodes of a second level of abstraction by performing abstraction by removing modules from an intermediate representation structure (520). The computing device (100) can generate an intermediate representation structure composed only of the second level of abstraction from an intermediate representation structure that reflects both the first level of abstraction and the second level of abstraction.

[0288] In one embodiment, when generating a target artificial intelligence model, the computing device (100) may generate a target artificial intelligence model from an intermediate representation structure (520) by increasing the level of abstraction. For example, an intermediate representation structure (520) to which abstraction of the first level of abstraction and abstraction of the second level of abstraction is applied may be converted into a target artificial intelligence model abstracted only at the first level of abstraction. For example, depending on the implementation mode, an intermediate representation structure (520) to which abstraction of the first level of abstraction is applied may be converted into a target artificial intelligence model to which abstraction of both the first level of abstraction and the second level of abstraction is applied. A technique according to one embodiment of the present disclosure may perform conversion or optimization of an artificial intelligence model by configuring a source artificial intelligence model of the first level, a source intermediate representation structure of the second level, a target intermediate representation structure of the second level, and a conversion flow to the target artificial intelligence model of the first level. In one embodiment, bidirectional conversion of the level of abstraction between the high level and the low level may be performed during the optimization or conversion process of the artificial intelligence model.

[0289] FIG. 6 exemplarily illustrates a method in which a source artificial intelligence model (610) is converted into a target artificial intelligence model (670) using intermediate representation structures (620, 650, 660) according to one embodiment of the present disclosure.

[0290] In one embodiment, a computing device (100) may receive a source artificial intelligence model (610) to be converted. The source artificial intelligence model (610) may include a plurality of operators (or layers) (610a and 610b). The computing device (100) may receive a source artificial intelligence model (610) to be optimized from a user terminal. For example, as the source artificial intelligence model (610) is parsed by the computing device (100), the operators (610a and 610b) of the source artificial intelligence model (610) may be identified. As another example, the operators (610a and 610b) of the source artificial intelligence model (610) may be identified using a database or lookup table that manages identification information of the source artificial intelligence model (610) and identification information of the operators included in the model.

[0291] In one embodiment, as operators (610a and 610b) in the source artificial intelligence model (610) are converted into modules (620a and 620b) containing nodes, a source intermediate representation structure (620) corresponding to the source artificial intelligence model (610) may be generated. The source intermediate representation structure (620) may represent the result of converting the source operators (610a and 610b) in the source artificial intelligence model (610) into modules (620a and 620b) containing nodes (625a, 625b, 625c and 625d). As described above, each of the nodes (625a, 625b, 625c and 625d) may correspond to a minimum unit of atomic operator that is no longer decomposed. In the example of FIG. 6, the source intermediate representation structure (620) may include a first module (620a) representing a first operator (610a) of the source artificial intelligence model (610) and a second module (620b) representing a second operator (610b). The nodes (625a and 625b) in the source intermediate representation structure (620) represent a set of atomic operators representing the first operator (610a) of the source artificial intelligence model (610) and may be determined independently of the framework (i.e., the framework of the source artificial intelligence model (610)). The first module (620a) in the source intermediate representation structure (620) may represent the first operator (610a) of the source artificial intelligence model (610). The first module (620a) in the source intermediate representation structure (620) may be configured dependently on the framework of the source artificial intelligence model (610). In order to preserve the connection relationships of the source artificial intelligence model (610), source modules (620a and 620b) corresponding to each of the source operators (610a and 610b) may be created within the source intermediate representation structure (620). Additionally, nodes (625a, 625b, 625c and 625d) that are not dependent on the framework may be created for the operators (or layers (610a and 610b)) of the source artificial intelligence model (610).A computing device (100) can generate a source intermediate representation structure (620) by configuring framework-dependent information for a source artificial intelligence model (610) into modules (620a and 620b) and configuring atomic operators corresponding to each of the source operators (610a and 610b) of the source artificial intelligence model (610) in a mapping table into nodes (625a, 625b, 625c and 625d). As described above, a mapping table defining which nodes can be converted for each layer or operator of a specific framework can be used to generate the source intermediate representation structure (620). Additionally, since additional information related to the nodes may also change depending on the type of layer or operator, information regarding these arguments may also be stored in the mapping table.

[0292] In one embodiment, the computing device (100) can convert a source intermediate representation structure (620) into a general-purpose intermediate representation structure (650:630 and / or 640). For example, the general-purpose intermediate representation structure (650) can serve as an intermediate step for converting the source intermediate representation structure (620) into a target intermediate representation structure (660). For example, the general-purpose intermediate representation structure (650) can be generated by a method of removing framework-dependent information from the source intermediate representation structure (620). For example, the general-purpose intermediate representation structure (650) can be generated by a conversion method of removing framework-dependent information from the source intermediate representation structure (620) and retaining framework-independent information. For example, the general-purpose intermediate representation structure (650:630 and / or 640) may also have a graph structure. For example, the universal intermediate representation structure (650:630 and / or 640) may mean an intermediate representation structure that converts the source intermediate representation structure (620) into a graph structure for universal use. For example, the universal intermediate representation structure (650:630 and / or 640) may mean an intermediate representation structure that converts the source intermediate representation structure (620) into a framework-independent structure.

[0293] In one embodiment, the universal intermediate representation structure (630) may be created by establishing a one-to-one correspondence between modules and nodes. In one embodiment, each of the modules (630a and 630b) of the universal intermediate representation structure (630) may be configured to include a single node. For example, a conversion from the source intermediate representation structure (620) to the universal intermediate representation structure (630) may be achieved by creating a single module that covers each of the nodes (625a, 625b, 625c and 625d) existing on the source intermediate representation structure (620). For example, a conversion from the source intermediate representation structure (620) to the universal intermediate representation structure (630) may be achieved by forming a single module for each of the nodes (625a, 625b, 625c and 625d) existing on the source intermediate representation structure (620). As illustrated in FIG. 6, each of the nodes (625a, 625b, 625c, and 625d) of the source intermediate representation structure (620) may correspond to each of the nodes (635a, 635b, 635c, and 635d) of the universal intermediate representation structure (630). In the process of converting the source intermediate representation structure (620) into the universal intermediate representation structure (630), the modules (620a and 620b) of the source intermediate representation structure (620) may be removed, and modules (630a, 630b, 630c, and 630d) corresponding to each of the nodes (635a, 635b, 635c, and 635d) of the universal intermediate representation structure (630) may be created.

[0294] In one embodiment, the universal intermediate representation structure (640) may mean an intermediate representation structure configured to include only nodes corresponding to atomic operators. For example, the universal intermediate representation structure (640) may have a form in which modules (620a and 620b) are removed from the source intermediate representation structure (620). The computing device (100) may generate the universal intermediate representation structure (640) by removing the framework-dependent modules (620 and 620b) from the source intermediate representation structure (620) and retaining the framework-independent nodes (625a, 625b, 625c and 625d). As illustrated in FIG. 6, the universal intermediate representation structure (640) may be configured to include the framework-independent nodes (640a, 640b, 640c and 640d). Each of the nodes (640a, 640b, 640c, and 640d) of the general intermediate representation structure (640) can correspond to each of the nodes (625a, 625b, 625c, and 625d) of the source intermediate representation structure (620).

[0295] In one embodiment, the computing device (100) may receive a target framework to be compiled and / or converted from a user. In one embodiment, the computing device (100) may convert a general-purpose intermediate representation structure (650) into a target intermediate representation structure (660). The target intermediate representation structure (660) may refer to an intermediate representation structure converted to be compatible with a framework corresponding to a target artificial intelligence model (670) that is the purpose of the conversion. The computing device (100) may receive target framework information from a user and may convert the general-purpose intermediate representation structure (650) into a target intermediate representation structure (660) using the received target framework information. The target intermediate representation structure (660) may be generated by creating target modules (660a, 660b, and 660c) corresponding to each of the target operators included in the target framework information. For example, a computing device (100) can identify modules (660a, 660b, and 660c) dependent on a target framework and identify one or more nodes to be included in each of the modules (660a, 660b, and 660c) dependent on the target framework. For example, a mapping table may be used in this identification process. For example, by using the mapping relationship between the combination of target modules (660, 660b, and 660c) and nodes stored in the mapping table, the node(s) to be included in the target modules (660, 660b, and 660c) are determined, and a target intermediate representation structure (660) can be generated accordingly. For example, three modules on the universal intermediate representation structure (630) can be converted into a single module (660a) on the target intermediate representation structure (660), and the target intermediate representation structure (660) can be configured such that Node #2, Node #3, and Node #4 on the universal intermediate representation structure (630) are included within the single module (660a).For example, a node (635a) in the universal intermediate representation structure (630) may correspond to a node (665a) in the target intermediate representation structure (660), a node (635b) in the universal intermediate representation structure (630) may correspond to a node (665b) in the target intermediate representation structure (660), a node (Node#4 (not shown)) in the universal intermediate representation structure (630) may correspond to a node (665c) in the target intermediate representation structure (660), a node (635c) in the universal intermediate representation structure (630) may correspond to a node (665d) in the target intermediate representation structure (660), and a node (635d) in the universal intermediate representation structure (630) may correspond to a node (665e) in the target intermediate representation structure (660). For example, the target intermediate representation structure (660) may be configured such that three nodes on the general intermediate representation structure (640) are included in a single module (660a) on the target intermediate representation structure (660). For example, the target intermediate representation structure (660) may be configured such that one node on the general intermediate representation structure (640) is included in a single module (660b) on the target intermediate representation structure (660). For example, the target intermediate representation structure (660) may be configured such that one node on the general intermediate representation structure (640) is included in a single module (660c) on the target intermediate representation structure (660).For example, a node (640a) in the universal intermediate representation structure (640) may correspond to a node (665a) in the target intermediate representation structure (660), a node (640b) in the universal intermediate representation structure (640) may correspond to a node (665b) in the target intermediate representation structure (660), a node (Node#4 (not shown)) in the universal intermediate representation structure (640) may correspond to a node (665c) in the target intermediate representation structure (660), a node (640c) in the universal intermediate representation structure (640) may correspond to a node (665d) in the target intermediate representation structure (660), and a node (640d) in the universal intermediate representation structure (640) may correspond to a node (665e) in the target intermediate representation structure (660).

[0296] In one embodiment, when converting from a general-purpose intermediate representation structure (650) to a target intermediate representation structure (660), the nodes within the general-purpose intermediate representation structure (650) are not removed, and the combination of the nodes into modules may be changed.

[0297] In one embodiment, nodes can be grouped by target module unit by using a method of assigning nodes (640a, 640b, 640c, and 640d) constituting a general-purpose intermediate representation structure (650) to each of the target modules (660a, 660b, and 660c). By this grouping, target modules (660a, 660b, and 660c) are created and a target intermediate representation structure (660) can be created.

[0298] In one embodiment, the computing device (100) can perform a direct conversion from a source intermediate representation structure (620) to a target intermediate representation structure (660). In this embodiment, a direct conversion between the source intermediate representation structure (620) and the target intermediate representation structure (660) can be performed without using a general-purpose intermediate representation structure (650). For example, the computing device (100) can determine which nodes each of the modules (660a and 660b) constituting the target intermediate representation structure (660) will include by using information regarding the target framework of the target artificial intelligence model (670). This determination can be made, for example, by using mapping relationships stored in a mapping table. The computing device (100) can perform a conversion from source modules (620a and 620b) to target modules (660a and 660b) dependent on the target framework by using the correlation between the combination of target operators and nodes stored in the mapping table.

[0299] In one embodiment, the computing device (100) may receive a target framework to be compiled and / or converted from a user. The computing device (100) may receive platform information from the user on which the artificial intelligence model will be deployed after compilation and / or conversion. Although the operators (e.g., layers) of various frameworks are defined with the same name, there are cases where the same dynamics do not proceed. For example, in the case of a specific framework, a bias may be included in the Conv operator, while in the case of another framework, a bias may not be included in the Conv operator. For example, depending on the target framework, the two nodes, Conv and bias, may be combined into a single Conv of the output framework or divided into a Conv module and an Add module, respectively. Information regarding the operator information for each framework and / or the configuration method of the intermediate representation structure for each framework may be provided in the mapping table described above.

[0300] In one embodiment, the computing device (100) can perform the creation and transformation of an intermediate representation structure using platform information received from a user. For example, the computing device (100) may remove or avoid operators that are not supported on a specific platform during the process of converting to a module. For example, if an operator that is not supported on a specific platform needs to be implemented in a different form, the computing device (100) may transform the operator into a corresponding operator within a mapping table or a compatible (replaceable) operator. If no replaceable operator exists, the module may be configured using node(s) representing the smallest unit of the operator.

[0301] In one embodiment, each module within the intermediate representation structure can be transformed not only into an operator (or layer) of the framework intended as output, but also into an operator compatible with (or executable on) the platform when the converted or compiled model is to be inferred on an actual platform. Accordingly, a technical effect can be derived in which platform optimization or hardware optimization of an artificial intelligence model can be achieved in an efficient manner using the intermediate representation structure. As such, the technique according to one embodiment of the present disclosure can have the technical effect of enabling optimization not only from a framework perspective but also from a hardware and runtime perspective.

[0302] In one embodiment, the computing device (100) can convert a target intermediate representation structure (660) into a target artificial intelligence model (670). In the process of converting to the target artificial intelligence model (670), a module (660a) may be converted into an operator (670a), a module (660b) may be converted into an operator (670b), and a module (660c) may be converted into an operator (670c). Such conversion between modules and operators may be performed, for example, using mapping relationships defined in a mapping table. Such conversion between modules and operators may correspond to the inverse of the way the source artificial intelligence model (610) is converted into a source intermediate representation structure (620).

[0303] A technique according to one embodiment of the present disclosure can generate an intermediate representation structure regardless of operators (layers) and modules that are organized differently between frameworks by using a minimum unit of operator called a node, which is independent of the framework. A technique according to one embodiment of the present disclosure can allow efficient conversion to operators (layers) of various frameworks by grouping or combining these nodes into modules corresponding to operators of various frameworks. Through this, it becomes possible to convert and compile into various frameworks from a graph structure composed of nodes, regardless of the input or output framework.

[0304] FIG. 7 illustrates, in accordance with one embodiment of the present disclosure, how model optimization tools (375, 380 and 385) are applied to an intermediate representation structure during the converting process of an artificial intelligence model.

[0305] In one embodiment, the computing device (100) can perform performance optimization of the converted target artificial intelligence model (300) by running model optimization tools (375, 380 and 385) during the process of converting the source artificial intelligence model (200) into the target artificial intelligence model (300). Through performance optimization, the execution performance of the target artificial intelligence model (300) on a specific platform and / or a specific framework can be improved.

[0306] In one embodiment, model optimization tools (375, 380 and 385) may be applied or executed on a novel intermediate representation structure according to one embodiment of the present disclosure.

[0307] In one embodiment, the computing device (100) can convert a source artificial intelligence model (300) into a source intermediate representation structure (710) using a loader manager (340). In one embodiment, the computing device (100) can determine a loader corresponding to the source artificial intelligence model (200) among a plurality of loaders for generating an intermediate representation structure using a loader manager (340), and convert the source artificial intelligence model (300) into a source intermediate representation structure (710) using the determined loader.

[0308] In one embodiment, the source intermediate representation structure (710) may include modules (710a, 710b, and 710c) corresponding to each source operator executable on a first framework associated with the source artificial intelligence model (200). Additionally, nodes included in each of the modules (710a, 710b, and 710c) may be determined using a mapping table representing the mapping relationship between model operators and atomic operators of the first framework supporting the source artificial intelligence model (200). A source intermediate representation structure (710) corresponding to the source artificial intelligence model (200) may be generated using these nodes (Node 1, Node 2, Node 3, Node 4, and Node 5) and modules (710a, 710b, and 710c). The modules (710a, 710b, and 710c) of the source intermediate representation structure (710) may be determined dependently on the first framework. The nodes included in each of the modules (710a, 710b, and 710c) of the source intermediate representation structure (710) can be determined dependently on the source operators of the source artificial intelligence model (200) corresponding to the first framework.

[0309] In one embodiment, the computing device (100) can convert (715) from a source intermediate representation structure (710) into a target intermediate representation structure (720) corresponding to a second framework that supports the target artificial intelligence model (300) to be converted. For example, the target intermediate representation structure (720) may be generated by a model editor (350), an export manager (360), and / or a separate module converter (not shown). Additionally, a universal intermediate representation structure, which is an intermediate representation independent of the framework, may be used in the process of converting the source intermediate representation structure (710) into the target intermediate representation structure (720). Such a universal intermediate representation structure may include, for example, a combination of nodes corresponding to a minimum unit of atomic operator that is universally applicable to a plurality of frameworks.

[0310] In one embodiment, the target intermediate representation structure (720) may include nodes identical to the nodes (Node 1, Node 2, Node 3, Node 4, and Node 5) included in the source intermediate representation structure (710). The modules (720a, 720b, and 720c) of the target intermediate representation structure (720) may be determined based on the second framework. The modules (720a, 720b, and 720c) of the target intermediate representation structure (720) may be determined dependently on the second framework. The nodes included in each of the modules (720a, 720b, and 720c) of the target intermediate representation structure (720) may be determined dependently on the target operators of the target artificial intelligence model (300) corresponding to the second framework. A set of nodes to be included in modules (720a, 720b, and 720c) can be determined within the range of nodes (Node 1, Node 2, Node 3, Node 4, and Node 5) included in the source intermediate representation structure (710). Based on the second framework, a set of nodes to be included in modules (720a, 720b, and 720c) can be determined within the range of nodes (Node 1, Node 2, Node 3, Node 4, and Node 5) included in the source intermediate representation structure (710). In this converting process, the target intermediate representation structure (720) can be configured so that Node 4, which was included in module (710b) of the source intermediate representation structure (710), is included in module (720c) of the target intermediate representation structure (720). Module conversion (715) can change the grouping of modules to fit a target framework capable of supporting the target artificial intelligence model (300).

[0311] In one embodiment, various model optimization tools (375, 380 and 385) are operated based on an intermediate representation structure, and the intermediate representation structure may be modified according to these operations, thereby generating a lightweight and / or optimized intermediate representation structure. Finally, a lightweight and / or optimized target artificial intelligence model (300) may be generated.

[0312] In one embodiment, various model optimization tools (375, 380, and 385) may be executed or applied on the target intermediate representation structure (720). In the example of FIG. 7, the model optimization tools are illustrated as being applied in the order of the compressor (375), the graph optimizer (380), and the quantizer (385); however, depending on the implementation mode, the model optimization tools may be applied in a different order, and only one or two of these model optimization tools may be used. Additionally, other model optimization tools for performance transformation of the artificial intelligence model, other than the compressor (375), the graph optimizer (380), and the quantizer (385), may be applied on the intermediate representation structure. Furthermore, in the example of FIG. 7, the model optimization tools are illustrated as being executed on the target intermediate representation structure (720); however, depending on the implementation mode, it is also possible for the model optimization tools to be executed on the source intermediate representation structure (720).

[0313] In one embodiment, the intermediate representation manager (330) of the computing device (100) may modify the graph of the intermediate representation structure and / or modify the information stored in the intermediate representation structure by working with model optimization tools. For example, the model editor (350) of the intermediate representation manager (330) may modify the intermediate representation structure using transformation, lightweighting, and / or optimization-related information received from the model optimization tools.

[0314] In one embodiment, it is also possible to implement model optimization tools directly performing modifications to intermediate representation structures.

[0315] In one embodiment, the intermediate representation structure is a graph structure composed of modules, nodes, and tensors, and for each component, in addition to information related to the graph structure, information required by model optimization tools may be included. The novel intermediate representation structure according to one embodiment of the present disclosure has a hierarchical or dependent structure of modules dependent on the framework and nodes independent of the framework, so that a method for processing modules during the operation of model optimization tools must be defined. Such a method for processing modules is stored in the intermediate representation structure (e.g., modules, nodes, and / or tensors), and lightweighting and / or optimization operations of model optimization tools can be performed by utilizing the stored information.

[0316] In one embodiment, the intermediate representation structure stores optimization reference information, so that the stored optimization reference information can be utilized when the model optimization tool operates. For example, depending on the type of model optimization tool, the optimization reference information used by the model optimization tool among the optimization reference information stored in the intermediate representation structure may differ. That is, there exists a correlation between the model optimization tool and the optimization reference information stored and used in the intermediate representation structure.

[0317] In one embodiment, as shown in the example of FIG. 7, when a plurality of model optimization tools are applied to an intermediate representation structure, the inputs and outputs of the model optimization tools may be connected sequentially. For example, an intermediate representation structure to which a first change is applied by one model optimization tool may be input to another model optimization tool, and a second change may be applied to the intermediate representation structure to which the first change is applied by the other model optimization tool.

[0318] In one embodiment, when the intermediate representation structure is changed as model optimization tools are applied to the intermediate representation structure, the hierarchical structure of the intermediate representation structure may be maintained. For example, when at least a part of the intermediate representation structure is changed, information of the module may be used as optimization reference information used by the model optimization tool. Accordingly, the hierarchical structure of the intermediate representation structure before and after the change of the intermediate representation structure by the model optimization tool may be maintained. In one embodiment, when the intermediate representation structure is changed as model optimization tools are applied to the intermediate representation structure, the dependency relationship with the framework of the intermediate representation structure (e.g., a second framework) may be maintained. For example, the computing device (100) may apply the model optimization tool to the intermediate representation structure using information of the target framework (e.g., a second framework) included in the intermediate representation structure. Accordingly, the dependency relationship of the module with the target framework (e.g., a second framework) before and after the change of the intermediate representation structure may be maintained.

[0319] In one embodiment, the intermediate representation structure to which model optimization tools are applied may have a graph structure comprising: a node, a module composed of one or more nodes, and a tensor representing the input of the module, the output of the module, the weight input to the module, and the activation input to the module on the intermediate representation structure. Optimization reference information to be used for the operation of the model optimization tool may be stored in the intermediate representation structure and utilized during the operation of the model optimization tool. The optimization reference information may include channel axis tracking information, information indicating whether compression is possible, target hardware information, target framework information, and / or quantization information. Different types of optimization reference information may be used depending on the type of model optimization tool. As an example, not a limitation, if the model optimization tool is a compressor (375), a first optimization reference information indicating a compressible operator among the operators of the artificial intelligence model within the intermediate representation structure may be used in the compressor (375). As an example, not limited to, where the model optimization tool is a graph optimizer (380), a second optimization reference information containing quantization information for determining whether to change a module of an intermediate representation structure within an intermediate representation structure may be used in the graph optimizer. As an example, not limited to, where the model optimization tool is a quantizer (385), a third optimization reference information containing target platform information on which the artificial intelligence model will be executed within an intermediate representation structure may be used in the quantizer (385).

[0320] In one embodiment, the computing device (100) may modify the target intermediate representation structure (720) using a compressor (375). For example, the compressor (375) may receive the intermediate representation structure (720) to be lightweighted and generate or output a lightweighted intermediate representation structure (730). For example, the compressor (375) may modify the target intermediate representation structure (720) to lightweight a target artificial intelligence model. A lightweighted target intermediate representation structure (730) may be generated by the compressor (375). For example, the computing device (100) may modify modules and / or nodes on the intermediate representation structure using the compressor (375). Channel axis information may be utilized during the modification process of the modules and / or nodes. For example, the compressor (375) may reduce the size of the operator. For example, the compressor (375) may reduce the size of the node on the intermediate representation structure. In FIG. 7, Node 2 in module (730a) is compressed and Node 3 in module (730b) is compressed as the compressor (375) is executed on an intermediate representation structure. For example, the compressor (375) can remove unnecessary nodes within the module. For example, the compressor (375) can reduce the size of the node by removing unnecessary parts within the node.

[0321] In one embodiment, the compressor (375) may support a model lightweighting method including pruning and filter decomposition. The computing device (100) may add a value regarding whether pruning is possible and / or whether filter decomposition is possible to a module corresponding to an operator (layer) capable of pruning or filter decomposition on an intermediate representation structure. For example, the computing device (100) may extract tag information specified for a compressible module or a compressible node from the intermediate representation structure and determine a compression configuration value according to the compressor (375). The computing device (100) may modify at least a part of the intermediate representation structure using the compressor (375) based on the tag information and the compression configuration value. For example, the computing device (100) may extract information about another module connected to a compressible module or information about another node connected to said compressible node from the intermediate representation structure and modify at least a part of the intermediate representation structure using the compressor (375) based on said extracted information. For example, if the compressor (375) performs a pruning operation, the tensor (information stored in the tensor) in the intermediate representation structure may be changed by the pruning operation. For example, if the compressor (375) performs a filter decomposition operation, the nodes and / or modules in the intermediate representation structure may be changed by the filter decomposition operation.

[0322] In one embodiment, the compressor (375) can perform platform-aware compression by utilizing platform information (e.g., hardware information). For example, the compressor (375) can perform lightweighting optimized for the corresponding hardware by utilizing the phenomenon where, when the number of output channels (Cout) of a neural network is increased on specific hardware, the inference latency does not increase linearly but remains constant in a specific section and then increases rapidly at a specific point.

[0323] In one embodiment, the graph optimizer (380) may modify at least a portion of the intermediate representation structure to increase the execution efficiency of the target artificial intelligence model (300). The graph optimizer (380) may generate an optimized target intermediate representation structure (740) including modified target modules (740a and 740b) by modifying the combination of nodes (Node 1, Node 2, Node 3, Node 4 and Node 5) included in the intermediate representation structure (730). A graph-optimized intermediate representation structure (740) may be generated by the graph optimizer (380). In the example of FIG. 7, a graph optimization operation is illustrated in which Node 1 is removed from the module (730a) of the intermediate representation structure (730) input to the graph optimizer (380). As a result of removing Node 1 from the module (730a), the module (740a) of the graph-optimized intermediate representation structure (740) may be modified to include only Node 2. In the example of FIG. 7, the result of integrating modules (730b) and modules (730c) of an intermediate representation structure (730) input to a graph optimizer (380) into a single module (740b) is illustrated. As the optimization operation of the graph optimizer (380) is applied to the intermediate representation structure, a module (740b) having a connection relationship of Node 3, Node 4, and Node 5 is created, and the existing modules (730b and 730c) can be removed from the intermediate representation structure.

[0324] In one embodiment, the graph optimizer (380) may delete at least one node among the nodes included in the intermediate representation structure, or move at least one node among the nodes included in the intermediate representation structure from a first target module to a second target module on the intermediate representation structure, or delete at least one module among the modules included in the intermediate representation structure and move the node included in the deleted module to another module. In one embodiment, the graph optimizer (380) may determine a target template corresponding to a combination pattern of modules on the intermediate representation structure from among pre-stored module combination templates, and by applying the target template to the intermediate representation structure, change at least a part of the intermediate representation structure in template units representing the combination of modules. Quantization information stored in the tensor of the intermediate representation structure may be used to determine whether to apply the target template to the intermediate representation structure.

[0325] In one embodiment, the graph optimizer (380) may replace a module included in an intermediate representation structure with another module. The graph optimizer (380) may change a combination of nodes in an intermediate representation structure to a combination of other nodes. For example, the Matrix Multiplication operator representing matrix multiplication may be replaced with a combination of basic operators (e.g., nodes) to which low-level abstraction is applied during the graph optimization process. For example, the Matrix Multiplication operator may be replaced with a combination of Reshape, Convolution, Split, and Concatenation. In this example, the module and its node corresponding to the Matrix Multiplication operator within the intermediate representation structure may be deleted, and new modules and nodes may be created.

[0326] In one embodiment, a quantizer (385) may be applied to an intermediate representation structure to reduce at least one of the size and computational amount of an artificial intelligence model. The quantizer (385) may receive an intermediate representation structure (740) as input and generate a quantized intermediate representation structure (750). Modules (750a) and (750b) of the quantized intermediate representation structure (750) correspond to modules (740a) and (740b) of the input intermediate representation structure (740), respectively. Through quantization (385), a quantization parameter (or quantization format) such as INT 8 in the example of FIG. 7 may be added to the quantized intermediate representation structure (750). For example, INT 8 represents a quantization parameter that represents the computation and parameters of the artificial intelligence model as an 8-bit integer. Accordingly, for example, since an artificial intelligence model can be represented by scaling a 32-bit floating-point (float 32) value into an 8-bit integer value, the size of the model can be reduced and the computation speed can be increased. In the example of FIG. 7, it is shown that the quantization parameter of float 32 is converted to INT 8 by a quantizer (385). For example, information about a quantization parameter or quantization format such as INT 8 can be stored in a tensor of an intermediate representation structure. In the example of FIG. 7, in the quantized intermediate representation structure (750), information about INT 8 can be stored in a tensor representing the input of module (750a), information about INT 8 can be stored in tensor(s) representing the output of module (750a) and the input of module (750b), and information about INT 8 can be stored in a tensor representing the output of module (750b). For reference, quantization information (e.g., information about Float 32) in other intermediate representation structures (710, 720, 730, and 740) can also be stored in their tensors.

[0327] In one embodiment, the quantizer (385) may determine a range of tensor values ​​for an intermediate representation structure based on target platform information on which the artificial intelligence model is to be executed. The quantizer (385) may determine quantization parameters (e.g., quantization scale, zero point, etc.) for quantizing the artificial intelligence model using the range of tensor values. The quantizer (385) may generate a quantized intermediate representation structure (750) by changing the tensor within the intermediate representation structure based on the quantization parameters. Based on module boundary information on the intermediate representation structure, the tensor to be quantized or the tensor to which quantization information is to be stored may be determined.

[0328] In one embodiment, the export manager (360) can generate a target artificial intelligence model (360) using an intermediate representation structure (750) to which model optimization tools are applied. The export manager (360) can convert the intermediate representation structure into a target artificial intelligence model using a mapping table that represents the correlation between target operators and nodes corresponding to the framework of the target artificial intelligence model (360) (e.g., a second framework). In the example of FIG. 7, a module (750a) can be converted into an operator (760b), and a module (750b) can be converted into an operator (760d).

[0329] In one embodiment, a computing device (100) (e.g., an export manager (360)) can generate quantization / dequantization operators (760a, 760c, and 760e) during the converting process using quantization information stored in a tensor of a quantized intermediate representation structure (750). These quantization-related operators can be added to be connected to the inputs and outputs of operators corresponding to the modules. These quantization-related operators can be added at the boundaries of each module. In this way, quantization-related information stored in a tensor on the intermediate representation structure can be converted into quantization-related operators placed before and after the operators of the artificial intelligence model.

[0330] A technique according to one embodiment of the present disclosure allows for efficient optimization or lightweighting operations to be performed on an intermediate representation structure through the linkage between optimization reference information stored in the intermediate representation structure and model optimization tool(s). In the present disclosure, optimization and lightweighting may be used interchangeably.

[0331] Figure 8 illustrates an exemplary converting process with a model preprocessing procedure applied.

[0332] Among the technical features in Fig. 8, the description of features that overlap with the technical features described above will be omitted and replaced with the technical features described above.

[0333] In one embodiment, the computing device (100) can convert a source artificial intelligence model (200) into an intermediate representation structure (840) using a loader manager (340). For example, the loader manager (340) of the computing device (100) can allow the generation of an intermediate representation structure (840) by a loader by identifying a first framework corresponding to the source artificial intelligence model (200) and determining a loader corresponding to the first framework. The intermediate representation structure (840) may include modules (840b, 840e, and 840h) corresponding to each of the source operators of the source artificial intelligence model (200). Quantization-related information of the source artificial intelligence model (200) may be represented as Q nodes (840c and 840f) and DQ nodes (840a, 840d, and 840g) on ​​the intermediate representation structure (840).

[0334] In one embodiment, the computing device (100) can remove quantization-related nodes (840a, 840c, 840d, 840f, and 840g) and include quantization-related information in the tensor by performing preprocessing (810) on an intermediate representation structure (840). This preprocessing (810) may be performed, for example, by a loader manager (340) and / or a model editor (350).

[0335] In one embodiment, as preprocessing (810) is performed, a source intermediate representation structure (850) corresponding to a source artificial intelligence model (200) may be generated. The source intermediate representation structure (850) may include modules (850a, 850b, and 850c) that are determined dependently by a first framework capable of supporting the source artificial intelligence model (200). Through preprocessing (810), information related to a module (840a) of the intermediate representation structure (840) prior to preprocessing (e.g., DQ Node 6) may be stored in the tensor of the input portion of the module (850a) of the source intermediate representation structure (850). The tensor of the output portion of the module (850a) of the source intermediate representation structure (850) and the tensor of the input portion of the module (850b) of the source intermediate representation structure (850) may be the same tensor. The information related to Q Node 7 associated with the module (840c) of the intermediate representation structure (840) and the information related to DQ Node 8 associated with the module (840d) of the intermediate representation structure (840) may be identical. Accordingly, the information related to Q Node 7 and DQ Node 8 may be removed all at once, and the corresponding information may be stored in the tensor(s) of the output and input parts. Depending on the implementation, the Q Node and DQ Node may not be configured as pairs, and a node configuration in the form of Q Node, Q Node, and DQ Node may exist on the intermediate representation structure (840). Additionally, since the Q Node plays the role of Re-quantization during the preprocessing process, it may be possible to leave it intact rather than deleting it. The information related to the module (840f) of the intermediate representation structure (840) (e.g., Q Node 9) may be stored in the tensor of the output part of the module (850b) of the source intermediate representation structure (850). Information related to the module (840g) of the intermediate representation structure (840) (e.g., DQ Node 10) can be stored in the tensor of the input part of the module (850c) of the source intermediate representation structure (850).As described above, preprocessing (810) may include storing quantization nodes and inverse quantization nodes in the intermediate representation structure in the tensor in the intermediate representation structure. Preprocessing (810) may include removing quantization nodes and inverse quantization nodes in the intermediate representation structure from the intermediate representation structure. Preprocessing (810) may include removing modules related to quantization nodes and inverse quantization nodes in the intermediate representation structure from the intermediate representation structure.

[0336] In one embodiment, a preprocessed intermediate representation structure (850) may be transformed through module transformation (820) to generate a target intermediate representation structure (860) comprising modules (860a and 860b) dependent on a second framework corresponding to a target artificial intelligence model (300). The target intermediate representation structure (860) may be configured to include modules (860a) and modules (860b) to correspond to target operators of the second framework. Through module transformation (820), the modules (850a, 850b, and 850c) of the intermediate representation structure (850) may be transformed into modules (860a and 860b) of the target intermediate representation structure (860). Even if module conversion (820) to a target intermediate representation structure (860) is performed, the nodes (Node 1, Node 2, Node 3, Node 4, and Node 5) of the intermediate representation structure (850) may be retained without being removed. The method of combining these nodes may change depending on the framework. The computing device (100) may generate target modules (860a and 860b) dependent on the second framework by grouping the nodes included in the source intermediate representation structure or the general intermediate representation structure using information about the second framework to which the source artificial intelligence model (200) is to be converted as a grouping condition. These target modules (860a and 860b) may correspond to target operators in the target artificial intelligence model.

[0337] In one embodiment, a model optimization tool (830) may be used based on a target intermediate representation structure (860). As the model optimization tool (830) is applied to the target intermediate representation structure (860), an optimized or lightweight intermediate representation structure (870) may be generated. An optimized or lightweight intermediate representation structure (870) may be generated as at least some of the nodes and / or modules on the target intermediate representation structure (860) are changed. An optimized or lightweight intermediate representation structure (870) may be generated as information stored in the tensors on the target intermediate representation structure (860) is changed. The optimized or lightweight intermediate representation structure (870) may be optimized or lightweight to include modules (870a and 870b).

[0338] In one embodiment, the export manager (360) can convert an intermediate representation structure (870) into a structure corresponding to a target artificial intelligence model (300). For example, the export manager (360) of the computing device (100) can identify a second framework corresponding to the target artificial intelligence model (300) and determine a converter corresponding to the second framework, thereby allowing the creation of the target artificial intelligence model by the converter.

[0339] In one embodiment, the computing device (100) may extract quantization-related information stored in a tensor on an intermediate representation structure (870) and add quantization-related nodes and / or modules on the intermediate representation structure using the extracted quantization-related information. The computing device (100) (e.g., export manager (360) and / or model editor (350)) may reconstruct the intermediate representation structure by adding Q / DQ modules (880a, 880c, 880 and 880f) based on the quantization information stored in the tensor of the intermediate representation structure (870). The reconstructed intermediate representation structure (880) may be composed of a module (880a) containing DQ Node 5, a module (880b) corresponding to module (870a), a module (880c) containing Q Node 7, a module (880d) containing DQ Node 8, a module (880e) corresponding to module (870b), and a module (880f) containing Q Node 9. The computing device (100) may convert the reconstructed intermediate representation structure (880) into a target artificial intelligence model (300) using a converter corresponding to the second framework (e.g., a converter for conversion to the second framework or a converter for conversion from the first framework to the second framework). The target artificial intelligence model (300) may be configured to include target operators (890a, 890b, 890c, 890d, 890e, and 890f) as in reference numeral 890. A module (880a) of the reconstructed intermediate representation structure (880) may correspond to a DQ operator (890a) in the target artificial intelligence model structure (890). A module (880b) of the reconstructed intermediate representation structure (880) may correspond to operator 1 (890b) in the target artificial intelligence model structure (890). A module (880c) of the reconstructed intermediate representation structure (880) may correspond to a Q operator (890c) in the target artificial intelligence model structure (890).A module (880d) of the reconstructed intermediate representation structure (880) may correspond to a DQ operator (890d) in the target artificial intelligence model structure (890). A module (880e) of the reconstructed intermediate representation structure (880) may correspond to operator 2 (890e) in the target artificial intelligence model structure (890). And a module (880f) of the reconstructed intermediate representation structure (880) may correspond to a Q operator (890f) in the target artificial intelligence model structure (890). The computing device (100) can generate a target artificial intelligence model (890) executable in the second framework by converting the target modules, whose configuration of the intermediate representation structure has been changed, into target operators (890a, 890b, 890c, 890d, 890e, and 890f) executable in the second framework. Depending on the implementation mode, reference numbers 890b and 890e may be defined as target operators, and reference numbers 890a, 890c, 890d, and 890f may be defined as quantization operators.

[0340] In one embodiment, when the target artificial intelligence model receives information about the target platform to be deployed, the information about the target platform may be stored in modules and tensors in an intermediate representation structure. The computing device (100) may generate the target artificial intelligence model (300) by using the information about the target platform in the intermediate representation structure to remove target modules corresponding to operators that are not supported on the target platform.

[0341] A technique according to one embodiment of the present disclosure can universally achieve conversion between deep learning frameworks by using a novel intermediate representation structure to compensate for the respective advantages and disadvantages of various deep learning frameworks or to use a runtime optimized for a specific device. Through a novel intermediate representation technique for conversion to various frameworks, a user can derive an artificial intelligence model optimized for various frameworks and various platforms using the same model. Furthermore, a technique according to one embodiment of the present disclosure can efficiently achieve model optimization or lightweighting along with conversion between deep learning frameworks by using a novel intermediate representation structure.

[0342] FIG. 9 illustrates an exemplary structural diagram for converting an artificial intelligence model according to one embodiment of the present disclosure.

[0343] Among the descriptions in Fig. 9, any descriptions that overlap with the descriptions above will be replaced with the descriptions above for the sake of brevity.

[0344] In one embodiment, the computing device (100) may receive a source artificial intelligence model (200) and platform information (305). For example, the intermediate representation manager (330) of the computing device (100) may receive the source artificial intelligence model (200) and platform information (305), and use the received source artificial intelligence model (200) and / or platform information (305) to generate an intermediate representation structure, perform optimization of the intermediate representation structure, and / or perform framework conversion from the source artificial intelligence model (200) to the target artificial intelligence model (300).

[0345] In one embodiment, a source artificial intelligence model (200) can be converted into a source intermediate representation structure (940) through a loader manager (340). The loader manager (340) can determine a loader dedicated to the source framework of the source artificial intelligence model (200). The determined loader can convert the source artificial intelligence model (200) into a source intermediate representation structure (940).

[0346] In one embodiment, a dimension transformation (910), a channel axis transformation (920), and / or a model transformation (930) may be performed on a source intermediate representation structure (940). As a result of such performance, a target intermediate representation structure (950) corresponding to a target framework may be generated. The target intermediate representation structure (950) is an intermediate representation structure in which at least a portion of the source intermediate representation structure (940) has been modified, and modifications on the intermediate representation structure may be performed by a model editor (350). For example, the dimension transformation (910), the channel axis transformation (920), and / or the model transformation (930) may be performed by components within the intermediate representation manager (330). In another example, the dimension transformation (910), the channel axis transformation (920), and / or the model transformation (930) may be performed by components existing outside the intermediate representation manager (330), and the transformation results may be reflected on the intermediate representation structure by communicating with the intermediate representation manager (330). In one embodiment, at least some of the dimension transformation (910), channel axis transformation (920), and / or model transformation (930) may be executed in parallel with each other. In one embodiment, the dimension transformation (910), channel axis transformation (920), and / or model transformation (930) may be performed in an execution order different from the execution order in FIG. 9. For example, depending on the embodiment, at least some of the operations 910, 920, and 930 may be omitted. For example, depending on the embodiment, at least some of the operations 910, 920, and 930 may be included in at least one of the operations 375, 380, 385, and 390 of FIG. 3.

[0347] In one embodiment, a computing device (100) (e.g., an intermediate representation manager (330)) may use a channel tracking algorithm to estimate a dimension representing a channel axis among the dimensions of tensors within an intermediate representation structure (940) and store the estimated dimension in the intermediate representation structure (940). The estimated dimension (e.g., channel axis tracking information) may be stored in a tensor of the intermediate representation structure (940).

[0348] In one embodiment, a dimension transformation (910) may be performed using channel axis tracking information. The dimension transformation (910) may include fixing or changing the dimension of each tensor of the intermediate representation structure (940) to a specific dimension using the channel axis tracking information. The dimension transformation (910) may include, for example, fixing or changing all tensors configured differently as 2, 3, 4, 5, and / or 6 dimensions to 4 dimensions. The dimension values ​​of the tensors are changed by the dimension transformation (910), and the changed dimension values ​​may be stored in the intermediate representation structure (940) (e.g., tensors).

[0349] In one embodiment, channel axis transformation (920) may be performed using channel axis tracking information. Channel axis transformation (920) may mean changing the axis constituting the channel according to a specific criterion (e.g., a setting value of an intermediate representation structure). Channel axis transformation (920) may mean changing the axis constituting the channel according to a specific criterion (e.g., a setting value of an intermediate representation structure) using channel axis tracking information. For example, if the input in the intermediate representation structure is configured in the form of channel first with B, C, H, W, and the output in the intermediate representation structure is configured in the form of channel last with B, H, W, C, a change in the position of the channel axis is required. Therefore, channel axis transformation (920) may be performed to unify the position of the channel axis between the input and output within the intermediate representation structure. The position of the channel axis changed according to the channel axis transformation (920) may be stored in the intermediate representation structure (940). The position of the channel axis changed according to the channel axis transformation (920) can be stored in the tensor(s) in the intermediate representation structure (940).

[0350] In one embodiment, the computing device (100) determines a combination of nodes in an intermediate representation structure (940) suitable for target operators corresponding to the target framework of the target artificial intelligence model (300), and can create target modules by combining the determined nodes through module transformation (930). By creating target modules, a target intermediate representation structure (950) can be created. The transformation and modification of the target intermediate representation structure (950) can be performed by a model editor (350) that modifies the intermediate representation structure.

[0351] In one embodiment, the computing device (100) may perform optimization or lightweighting on a target intermediate representation structure (950) using a compressor (375), a quantizer (380), and a graph optimizer (385). The computing device (100) may perform optimization or lightweighting on a target intermediate representation structure (950) based on platform information (305). The platform information (305) may be used in the operation of the compressor (375), the quantizer (380), and / or the graph optimizer (385). For example, hardware constraints and / or compiler constraints in the platform information (305) may be used in the optimization or lightweighting operation of the compressor (375), the quantizer (380), and / or the graph optimizer (385).

[0352] In one embodiment, the intermediate representation structures (940 and 950) may be used to generate expected inference simulation results when the target artificial intelligence model (300) is executed on the platform prior to conversion to the target artificial intelligence model (300). The simulator (390) may generate inference simulation results without execution of the actual artificial intelligence model on the platform by using the intermediate representation structures (940 and 950) and platform information (305).

[0353] In one embodiment, the computing device (100) can execute a model optimization tool on a target intermediate representation structure (950) using the execution order and parameters of the model optimization tools on a target transformation scenario determined by the scenario manager (310).

[0354] In one embodiment, the computing device (100) may execute the model optimization tools sequentially on the target intermediate representation structure (950) according to the execution order of the model optimization tools on the target transformation scenario. Depending on the implementation mode, for model optimization tools designated as capable of parallel execution on the target transformation scenario, the computing device (100) may also be able to execute them in parallel on the target intermediate representation structure (950).

[0355] In one embodiment, the computing device (100) can use a simulator (390) to generate expected simulation results when a target transformation scenario is applied to a target artificial intelligence model. In this way, without performing actual platform and model execution, the utility of the determined target transformation scenario, the utility of the framework-transformed model, and / or the utility of the model optimization tool can be verified in a resource-efficient manner through the expected simulation results. As the module(s) dependent on the framework of the intermediate representation structure are modified according to the target platform, a simulation of the target artificial intelligence model on the target platform can be performed. In this way, the expected execution performance according to the combination of the target artificial intelligence model and the target platform when the model optimization tool is applied to the target artificial intelligence model can be verified. Accordingly, the utility of the model optimization tool can be verified.

[0356] In one embodiment, the computing device (100) may not execute the converted target AI model on the target platform, but may apply and / or execute the target model optimization tools included in the target conversion scenario to the target AI model through the simulator (390) according to the order specified in the target conversion scenario. Execution through the simulator (390) may include the application or execution of the target model optimization tools on an intermediate representation structure corresponding to the combination of the target AI model and the target platform. The computing device (100) may generate performance verification results including expected inference simulation results for when the converted target AI model is executed on the target platform. The performance verification results may be utilized in the verification process of the target conversion scenario.

[0357] In one embodiment, the computing device (100) can compare the performance verification result of the converted target artificial intelligence model with the performance prediction value mapped to the target conversion scenario. For example, the computing device (100) can compare the result simulated through a simulator with the performance prediction value stored in the scenario DB to verify whether the actual performance of the target conversion scenario differs from the prediction value. Based on the result of the comparison, the computing device (100) can decide whether to modify the target conversion scenario. For example, if the difference between the performance prediction value and the performance verification result obtained through the simulator exceeds a predetermined threshold difference (or if the performance is not as good as the performance prediction value), the computing device (100) may decide to modify the target conversion scenario. In this case, the computing device (100) may change at least one parameter among the target model optimization tools included in the target conversion scenario. For example, changing a parameter may mean changing the value of an optimization-related parameter (e.g., compression rate, compression ratio, compression method, quantization method, etc.) mapped to the model optimization tools. The computing device (100) can perform a second transformation of the target artificial intelligence model by applying target model optimization tools containing changed parameters to the target artificial intelligence model prior to transformation. Accordingly, the computing device (100) can achieve a technical effect of achieving more efficient and more accurate optimization of the artificial intelligence model by modifying the transformation results that did not reach the performance prediction values ​​in the scenario DB. For example, the second transformation is a transformation that follows the first transformation (i.e., the application of the first model optimization tool(s)) that was simulated through a simulator, and can be applied to the target artificial intelligence model prior to the first transformation. For example, the result of the second transformation can also be simulated by the simulator in the manner described above.If the simulation result for the second transformation satisfies a predetermined threshold criterion, the computing device (100) can update the scenario DB with the parameter values ​​included in the second transformation. If the simulation result for the second transformation satisfies a predetermined threshold criterion (e.g., falls within the range of the expected performance value, or does not deteriorate below the expected performance value), the computing device (100) can update the parameter values ​​of the target transformation scenario in the scenario DB with the parameter values ​​included in the second transformation.

[0358] In one embodiment, it is assumed that the performance verification result according to the first transformation is output as a 0.5% reduction in accuracy and a 3% reduction in inference speed according to the compressor parameters (e.g., compression ratio 10%) and quantizer parameters (e.g., Int8:Int16 = 1:9) included in the determined target transformation scenario. If the performance prediction value mapped to the target transformation scenario in the scenario DB is recorded as a 0.3% reduction in accuracy and a 3% reduction in inference speed, the computing device (100) determines that the performance verification result according to the first transformation does not satisfy the threshold criteria, changes at least one of the compressor and / or quantizer parameters included in the target transformation scenario, and can perform a second transformation of the target artificial intelligence model according to the target transformation scenario to which the changed parameters are applied. If the computing device (100) performs a simulation of the target artificial intelligence model to which the second transformation has been performed and determines that the simulation result satisfies the criteria according to the performance prediction value, it can output the target artificial intelligence model transformed or optimized according to the second transformation. In this case, the target transformation scenario can be updated with parameters defined by the second transformation. As the scenario DB is dynamically updated in this way, a scenario DB composed of more robust and robust transformation scenarios can be created and managed.

[0359] In one embodiment, the use of the simulator (390) may be associated with performance improvement of the scenario manager (310). Since the scenario DB can be dynamically updated based on performance measured through the simulator (390), the reliability and accuracy of the scenario DB can be improved.

[0360] In one embodiment, the computing device (100) can convert a target intermediate representation structure (950) into a target artificial intelligence model (300) using an export manager (360). A converting process can be performed to change each of the target modules of the target intermediate representation structure (950) into a target operator.

[0361] FIG. 10 is a flowchart illustrating, exemplarily, a method for outputting a target artificial intelligence model to which performance transformation has been applied.

[0362] Depending on the implementation mode, additional steps may be added to the flowchart of FIG. 10, or some steps within the flowchart may be omitted. Features of FIG. 10 that overlap with the features described above will be omitted in the description of FIG. 10 and replaced with the features described above.

[0363] In one embodiment, the computing device (100) may receive an intermediate representation structure that represents a target artificial intelligence model subject to performance conversion as a node corresponding to an atomic operator independent of the framework and a module dependent on the target framework of the target artificial intelligence model (1010).

[0364] In one embodiment, the intermediate representation structure may have a graph structure comprising modules dependent on a specific framework of a specific artificial intelligence model, nodes constituting said modules, and tensors. The graph structure may represent an artificial intelligence model, the modules may represent operators within the artificial intelligence model, the nodes may represent atomic operators constituting the operators of the artificial intelligence model, and the tensors may include information for the execution of the artificial intelligence model, modules, and / or nodes. For example, the tensors may represent the inputs of the modules, the outputs of the modules, weights input to the modules, and activations input to the modules on the intermediate representation structure. The intermediate representation structure may include optimization reference information to be used in the optimization operations of a model optimization tool. The optimization reference information may include channel axis tracking information, information indicating compressibility, target hardware information, target framework information, and quantization information.

[0365] In one embodiment, the computing device (100) can change at least a part of the intermediate representation structure by applying a model optimization tool to the intermediate representation structure (1020).

[0366] In one embodiment, the model optimization tool(s) can modify the model using the hierarchical structure of the intermediate representation structure. In one embodiment, the model optimization tool(s) can modify the intermediate representation structure using the hierarchical structure of the intermediate representation structure.

[0367] In one embodiment, when the computing device (100) changes the intermediate representation structure, it can determine the target of change in the intermediate representation structure by using information of the module of the intermediate representation structure. For example, the computing device (100) can determine the object to be changed among the nodes, modules, and tensors within the intermediate representation structure by using information of the module of the intermediate representation structure. Optimization or lightweighting operations of model optimization tools can be performed on the target of change.

[0368] In one embodiment, the intermediate representation structure is composed of graphs, nodes, and tensors and may further include modules that logically group the nodes. Modules in the intermediate representation structure can be conceptually used to output a model transformed to fit the framework. The presence or absence of modules does not alter the connectivity between the nodes and tensors constituting the graph in the intermediate representation structure. Modification of nodes and / or tensors within the intermediate representation structure is required to perform optimization by model optimization tools. Information related to modules may be required to maintain the hierarchical structure and dependencies of the intermediate representation structure during the process of modifying the intermediate representation structure by such model optimization tools. For example, a compressor may determine the nodes and / or tensors to be modified using parameters indicating compressibility, such as whether pruning is possible for each module within the intermediate representation structure. For example, a graph optimizer may determine the nodes to be replaced or changed through pattern matching of modules. For example, a quantizer may determine the tensors to be quantized based on modules. In these examples, the computing device (100) can determine an object in an intermediate representation structure that is the target of optimization for model optimization tools by using information of a module in an intermediate representation structure.

[0369] In an additional embodiment, the compressor may determine whether to prun (compress) on a node-by-node basis using parameters indicating whether compression is possible, such as whether pruning is possible per node.

[0370] The model optimization tool in the present disclosure may be used interchangeably with a model optimization algorithm, a model optimization method, and / or a model optimization method. As such a model optimization tool is applied to an intermediate representation structure by a computing device (100) (e.g., a server, etc.), a change in the performance of the model may be achieved.

[0371] In one embodiment, when a model optimization tool (e.g., a graph optimizer) modifies an intermediate representation structure, information about the target framework included in the intermediate representation structure may be used by the model optimization tool, and accordingly, the dependency relationship of the module to the target framework before and after the modification of the intermediate representation structure (e.g., the dependency relationship between the module and the node) may be maintained. In one embodiment, when a model optimization tool (e.g., a graph optimizer) modifies an intermediate representation structure, information about the module of the intermediate representation structure is used as optimization reference information used by the model optimization tool, so that the hierarchical structure of the intermediate representation structure before and after the modification (e.g., the hierarchical structure between the module and the node) may be maintained. In an additional embodiment, when a compressor modifies a channel, the dependency relationship and hierarchical structure within the intermediate representation structure are not changed. In an additional embodiment, as the quantization result of the quantizer is stored in a tensor, the dependency relationship and hierarchical structure within the intermediate representation structure are not changed.

[0372] In one embodiment, a model optimization tool may perform modifications to an intermediate representation structure using different optimization reference information from an intermediate layer structure. Depending on the type of model optimization tool, the optimization reference information used by the model optimization tool among the optimization reference information stored in the intermediate representation structure may differ. For example, if the model optimization tool i...

Claims

1. A method for converting a source artificial intelligence model into a target artificial intelligence model for framework conversion performed by a computing device, A step of receiving the above source artificial intelligence model executable in the first framework; A step of converting source operators included in the above source artificial intelligence model into a source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework; A step of generating a target intermediate representation structure including target modules dependent on the second framework and nodes independent of the framework, using information about the second framework to which the source artificial intelligence model is to be converted; and A step of generating a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into target operators by converting the target modules of the target intermediate representation structure into target operators executable in the second framework; Includes, and The source modules and target modules are composed of combinations of the nodes, and each of the nodes corresponds to an atomic operator independent of the framework, method.

2. In Paragraph 1, The step of converting the above source operators into the above source intermediate representation structure including source modules dependent on the first framework and nodes independent of the framework, A step of generating a source intermediate representation structure that represents the source artificial intelligence model in terms of the source module and the node by converting each of the source operators constituting the source artificial intelligence model into a source module composed of the nodes independent of the framework; Includes, and The above source modules correspond to the source operators dependent on the above first framework, method.

3. In Paragraph 1, A step of converting the above source intermediate representation structure into a universal intermediate representation structure for conversion to another framework different from the above first framework; Includes more, The step of generating the above target intermediate representation structure is, A step of converting the general-purpose intermediate representation structure into a target intermediate representation structure including the target modules dependent on the second framework and the nodes independent of the framework, using information about the second framework to which the source artificial intelligence model is to be converted; Includes, and The above general-purpose intermediate representation structure is, As an intermediate representation independent of the framework, comprising a combination of nodes corresponding to a minimum unit of atomic operator universally applicable to multiple frameworks, method.

4. In Paragraph 3, The step of converting the above source intermediate representation structure into the above universal intermediate representation structure is: In the above source intermediate representation structure, by removing the source modules dependent on the first framework and maintaining the nodes independent of the framework, or, By modifying the source intermediate representation structure so that a single node is included in a single source module, Generating the above general-purpose intermediate representation structure, method.

5. In Paragraph 3, The step of converting the above-mentioned general-purpose intermediate representation structure into the above-mentioned target intermediate representation structure including the above-mentioned target modules dependent on the above-mentioned second framework and the above-mentioned nodes independent of the framework, A step of grouping the nodes included in the general-purpose intermediate representation structure using information about the second framework to be converted from the source artificial intelligence model as a grouping condition; and A step of generating the target modules dependent on the second framework according to the result of the grouping above; Includes, and The above target modules correspond to the above target operators, method.

6. In Paragraph 1, The step of converting the above source operators into the above source intermediate representation structure is, A step of identifying combinations of nodes that match the combination of the source operator of the source artificial intelligence model and the first framework using a lookup table that identifies combinations of nodes that match the combination of the type of framework and the type of operator; and A step of converting each of the source operators constituting the source artificial intelligence model into the source module by configuring the combination of the identified nodes into a single module; including, method.

7. In Paragraph 1, The step of generating the above target intermediate representation structure is, A step of determining a recombination method of the nodes for compatibility of the second framework using a lookup table representing a transformation relationship between the target operators of the second framework and the nodes; and A step of generating the target modules dependent on the second framework by grouping the nodes using the determined recombination method; including, method.

8. In Paragraph 1, The step of generating the above target artificial intelligence model is, A step of generating a target artificial intelligence model compatible with the second framework by converting the arguments of the nodes included in the target modules into the arguments of the target operators of the second framework using a lookup table representing the conversion relationship between the second framework and the nodes and the arguments used for the operator to operate in the second framework; including, method.

9. In Paragraph 1, The step of converting the above source operators into the above source intermediate representation structure is, A step of converting the source operators included in the source artificial intelligence model into combinations of the nodes using a lookup table that uses framework-specific operators as keys and nodes corresponding to atomic operators as values; including, method.

10. In Paragraph 1, The step of generating the above target artificial intelligence model is, A step of generating a quantization operator and a dequantization operator corresponding to the second framework using quantization information of a tensor included in the target intermediate representation structure; Includes, and When converting the target modules of the above target intermediate representation structure into target operators executable in the second framework, the quantization operator and the inverse quantization operator are added to the target operators. method.

11. In Paragraph 1, A step of identifying the mode of the received source artificial intelligence model; Includes more of, and If the received source artificial intelligence model is identified as a first mode in which the model is executed after representing the operation as a graph, a step of converting the received source artificial intelligence model into the source intermediate representation structure is performed, and If the received source AI model is identified as a second mode in which the model is executed in units of operations without representing operations as a graph, a model optimization tool is applied to the source AI model, and a step of converting the source AI model to which the model optimization tool is applied into the source intermediate representation structure is performed, and The above model optimization tool comprises at least one of a compressor, a graph optimizer, and a quantizer. method.

12. In Paragraph 1, When the above target artificial intelligence model receives information about the target hardware to be deployed, the information about the target hardware is stored in the target module and tensor in the target intermediate representation structure, and The step of generating the above target artificial intelligence model is, A step of generating the target artificial intelligence model by removing a target module or node corresponding to an operator not supported by the target hardware using information of the target hardware in the target intermediate representation structure; including, method.

13. In Paragraph 1, When the above target artificial intelligence model receives information about the target hardware and target compiler to be deployed, a node supported by the target hardware and the target compiler and a module including said supported node are added to the target intermediate representation structure, or a node not supported by the target hardware and the target compiler and a module including said unsupported node are deleted from the target intermediate representation structure. method.

14. A computer-readable storage medium for storing a computer program, wherein, when the computer program is executed by a computing device, the computing device is enabled to perform an operation to convert a source artificial intelligence model into a target artificial intelligence model for framework conversion, and the operation is: The operation of receiving the above source artificial intelligence model executable in the first framework; An operation of converting source operators included in the above-mentioned source artificial intelligence model into a source intermediate representation structure including source modules dependent on the above-mentioned first framework and nodes independent of the framework; The operation of generating a target intermediate representation structure including target modules dependent on the second framework and nodes independent of the framework, using information about the second framework to which the source artificial intelligence model is to be converted; and An operation to generate a target artificial intelligence model in which the source operators of the source artificial intelligence model are converted into the target operators by converting the target modules of the target intermediate representation structure into target operators executable in the second framework; Includes, and The source modules and target modules are composed of combinations of the nodes, and each of the nodes corresponds to an atomic operator independent of the framework, Computer-readable storage media.

15. A computing device for converting a source artificial intelligence model into a target artificial intelligence model for framework conversion, The above computing device includes at least one processor, and The above at least one processor is: Receive the above source artificial intelligence model executable in the first framework; The source operators included in the above-mentioned source artificial intelligence model are converted into a source intermediate representation structure including source modules dependent on the above-mentioned first framework and nodes independent of the framework; Using information about a second framework to which the above source artificial intelligence model is to be converted, a target intermediate representation structure is generated that includes target modules dependent on the second framework and the nodes independent of the framework; and By converting the target modules of the above target intermediate representation structure into target operators executable in the second framework, a target artificial intelligence model is generated in which the source operators of the source artificial intelligence model are converted into the target operators, and The source modules and target modules are composed of combinations of the nodes, and each of the nodes corresponds to an atomic operator independent of the framework, Computing device.

Images