Model generation device and model generation method
By generating and testing model patterns on edge devices, the solution ensures only operable models are selected for machine learning, enhancing efficiency in developing models for edge devices.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- GILIA CO LTD
- Filing Date
- 2023-07-25
- Publication Date
- 2026-07-29
AI Technical Summary
Existing technologies fail to verify whether neural network models generated for edge devices actually work on the devices, leading to unnecessary processing and inefficient development.
Generate multiple model patterns by changing network hierarchy, graph format, and numerical format, and execute inference runtimes to determine operability on edge devices, selecting only operable models for machine learning.
Efficiently develops models applicable to edge devices by focusing on models that actually operate, reducing unnecessary processing and improving development efficiency.
Smart Images
Figure 0007896219000001 
Figure 0007896219000002 
Figure 0007896219000003
Abstract
Description
Technical Field
[0001] The present invention relates to a model generation device and a model generation method, and particularly to a device and a method for generating a model to be mounted on an edge device.
Background Art
[0002] In recent years, the movement to utilize AI (Artificial Intelligence) using machine learning has been on the rise. The development of technology for controlling the operations of various devices such as home appliances and industrial devices based on the results inferred by a model created by machine learning by combining AI-related technologies and IoT (Internet of Things)-related technologies has also progressed. An end device connected to a network is called an edge device, and by mounting AI on the edge device, it becomes possible to make instantaneous judgments and controls at the site where data is generated.
[0003] Generally, in order to perform machine learning to create a model, it is necessary to collect a large amount of learning data. Also, in order to create an appropriate model that suits the purpose, it is necessary to improve the model through many trials. Generally, the number of trial and error repetitions until an optimal model is obtained for an edge device becomes enormous and requires a lot of development time. Conventionally, for such problems, a technique has been known that narrows down candidates for a neural network to be the execution target of machine learning so that an optimal model can be efficiently obtained (for example, see Patent Document 1).
[0004] The neural network construction device described in Patent Document 1 determines candidate hyperparameters for the neural network using a first condition relating to the computing resources of the embedded device (such as hardware execution environment conditions like CPU operating frequency and memory size, the size of input data to the model and the size of output data from the model, and the accuracy target for inference by the model), and generates a neural network model using the optimal candidate hyperparameters searched from the candidate list using Bayesian optimization or the like. Then, it is determined whether the generated model satisfies a second condition relating to the performance the model should possess (such as performance constraints like target processing time), and models that do not satisfy the performance constraints are discarded, while models that do satisfy the performance constraints are subjected to machine learning using training data, the prediction accuracy of the model is evaluated, and then added to the list of evaluated models.
[0005] According to Patent Document 1, by generating a neural network that satisfies the first condition, it is possible to efficiently narrow down the candidates for the optimal neural network to one with a configuration that satisfies the resource conditions determined according to the problem to be solved. Furthermore, by adopting only the models that satisfy the second condition from among the narrowed-down models as the models to be trained, it is possible to filter out models that do not have the performance required for the application, even if they satisfy the scale constraints, and to narrow down the models to be performed on for machine learning and prediction accuracy evaluation. [Prior art documents] [Patent Documents]
[0006] [Patent Document 1] Patent No. 6799197 [Overview of the project] [Problems that the invention aims to solve]
[0007] However, the technology described in Patent Document 1 above determines candidate hyperparameters for the neural network to satisfy the first condition, but it does not verify whether the model generated using these candidate hyperparameters actually works on the edge device. As a result, models that do not actually work on the edge device may be subjected to machine learning and prediction accuracy evaluation after evaluation based on the second condition, leading to the problem of performing unnecessary processing.
[0008] This invention was made to solve these problems, and aims to enable the more efficient development of models applicable to edge devices by focusing machine learning on models that actually run on edge devices. [Means for solving the problem]
[0009] To solve the above-mentioned problems, the present invention generates multiple model patterns by changing the model components, which include at least one of the network hierarchy, graph format, channel order format, and numerical format. An inference runtime is then executed on the generated multiple model patterns to determine whether they can operate on edge devices, and the operable models are selected as candidate models for machine learning. [Effects of the Invention]
[0010] According to the present invention configured as described above, only the models that are determined to be operable on edge devices from among the multiple generated model patterns are selected as candidate models for machine learning. Therefore, machine learning can be limited to models that actually operate on edge devices, making it possible to develop models applicable to edge devices more efficiently. [Brief explanation of the drawing]
[0011] [Figure 1] This block diagram shows an example of the basic functional configuration of the model generation device according to this embodiment. [Figure 2] This flowchart shows an example of the operation of the model generation device according to this embodiment. [Figure 3] This is a block diagram showing an example of the functional configuration of a model generation device related to an application example. [Figure 4] This block diagram shows another example of the functional configuration of the model generation device according to this embodiment. [Modes for carrying out the invention]
[0012] Hereinafter, one embodiment of the present invention will be described with reference to the drawings. Figure 1 is a block diagram showing a basic example of the functional configuration of the model generation device 10 according to this embodiment. As shown in Figure 1, the model generation device 10 of this embodiment comprises a model generation unit 1 and a model verification unit 2 as its functional configuration. The model verification unit 2 comprises, specifically, an operation feasibility determination unit 11, an original model inference execution unit 12, and an error determination unit 13 as its functional configuration.
[0013] The above-mentioned functional blocks 1 and 2 can be configured using either hardware or software. For example, the above-mentioned functional blocks 1 and 2 can be realized by the operation of programs stored in storage media such as RAM, ROM, hard disk, or semiconductor memory, under the control of a microcomputer configured with a CPU, RAM, ROM, etc. A GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or DSP (Digital Signal Processor) may be used instead of or in addition to the CPU.
[0014] The model generation unit 1 generates multiple model patterns by changing the model components. The model components include at least one of the following: the hierarchical structure of the network, the format of the graph, the format of the channel order, and the format of the numerical values.
[0015] The hierarchical structure of a network is a structure relating to the combination, order, and size of the layers that make up the model, and includes at least one of the following structures: convolutional layers, pooling layers, transformer layers, attention layers, normalization layers, MLP layers, fully connected layers, layer composite block layers composed of these layers, activation functions, how layers are connected and aggregation layers, the configuration of the backbone, the configuration of the neck (e.g., deconvolution layers, upsample layers, feature pyramid layers, etc.), and the configuration of the head.
[0016] The graph format is the format of the model's computation graph, including various frameworks such as ONNX, PyTorch, Tensorflow, TFLite, Keras, OpenVINO, and scikit-learn. The channel order format is the format of the channel order used in the model's computation, for example, if the data handled by the model is images, it includes information such as the number of images, the number of channels (color, etc.), and the height and width of the images. The numerical format is the format of the numerical values used for the model's weights or computation, including information such as fp32, fp16, bfloat16, and int8.
[0017] The model generation unit 1 generates multiple model patterns (hereinafter referred to as "converted models") by converting the model components of an original model, for example, based on an original model generated by a predetermined combination of model components. The original model may be generated in advance by a device other than the model generation device 10, or it may be generated by the model generation unit 1.
[0018] The Model Verification Unit 2 verifies for each of the multiple transformation models generated by the Model Generation Unit 1 whether it can operate on an edge device and whether the performance error compared to the original model is within a certain range. Then, it selects only the transformation models that have been confirmed to be operable on an edge device and whose performance error compared to the original model is within a certain range as candidate models for machine learning.
[0019] The operation feasibility determination unit 11 of the model verification unit 2 determines the feasibility of operation on the edge device by executing the inference runtime for the conversion models of multiple patterns generated by the model generation unit 1, and selects an operable model as a candidate model for machine learning. Strictly speaking, an operable model is selected as a candidate model for determination by the error determination unit 13. The types of inference runtimes include those obtained by removing the development function from software with both model development and execution functions and extracting only the effective functions, as well as inference runtimes based on device emulators and inference runtimes that deploy and execute the model on evaluation hardware.
[0020] Inference runtimes are prepared for each type of various frameworks and various edge devices. The operation feasibility determination unit 11 uses an inference runtime that matches the conversion model among multiple types of inference runtimes, executes inference using the pre-prepared verification data, and verifies whether the conversion model is operable. Here, when no output can be obtained from the inference runtime, the operation feasibility determination unit 11 determines that the conversion model is inoperable. Also, when the inference runtime has a check function, the operation feasibility determination unit 11 can determine the feasibility of operation based on whether an error is output by that check function. When a normal output rather than an error is obtained from the inference runtime, the operation feasibility determination unit 11 determines that the conversion model is operable. In addition, when the format of the channel order or the format of the numerical values has changed due to the conversion process from the original model to the conversion model, the verification data input to the conversion model is adjusted to match the format of the data input to the conversion model.
[0021] In addition, when there is no inference runtime that matches the conversion model, the model conversion may be attempted using a dedicated conversion tool provided by the edge device manufacturer. If the conversion is successful, it is determined to be operable, while if the conversion fails, it is determined to be inoperable.
[0022] The original model inference execution unit 12 executes the inference runtime for the original model used by the model generation unit 1 for conversion. Here, the original model inference execution unit 12 uses an inference runtime compatible with the original model and executes inference using the same verification data as that used by the operation availability determination unit 11. In the case where the format of the channel order or the format of the numerical values has changed due to the conversion process from the original model to the conversion model, the output data from the inference runtime used for the inference of the original model is adjusted to match the format of the data output by the conversion model.
[0023] The error determination unit 13 determines whether the error (e.g., mean squared error) between the response information output from the inference runtime executed for the conversion model determined to be operable by the operation availability determination unit 11 and the response information output from the inference runtime executed for the original model by the original model inference execution unit 12 is less than or equal to a threshold value, and selects, as candidate models for machine learning, the conversion models for which the error is less than or equal to the threshold value among the conversion models determined to be operable.
[0024] FIG. 2 is a flowchart showing an operation example of the model generation apparatus 10 according to the present embodiment configured as described above.
[0025] First, the model generation unit 1 generates a conversion model by converting the model components of the original model (step S1). Next, the operation availability determination unit 11 of the model verification unit 2 executes the inference runtime for the conversion model generated by the model generation unit 1 (step S2) and determines whether the conversion model can operate on the edge device (step S3). Here, if it is determined that the operation is impossible, the conversion model generated in step S1 is discarded (step S7), and the process proceeds to step S8. Discarding the conversion model means excluding it from the processing target of machine learning.
[0026] On the other hand, if it is determined that the model is operational, the original model inference execution unit 12 uses an inference runtime that is compatible with the original model and performs inference using the same verification data as the operational feasibility determination unit 11 (step S4). Then, the error determination unit 13 determines whether the error between the response information output from the inference runtime executed for the transformed model and the response information output from the inference runtime executed for the original model is below a threshold (step S5).
[0027] If it is determined that the error is not below the threshold, the transformation model generated in step S1 is discarded (step S7), and the process proceeds to step S8. On the other hand, if it is determined that the error is below the threshold, the error determination unit 13 selects the transformation model generated in step S1 as a candidate model for machine learning (step S6). The process then proceeds to step S8. In step S8, it is determined whether or not to terminate the process. Here, it is determined whether or not to generate a different transformation model. If a different transformation model is to be generated, the process returns to step S1. On the other hand, if a different transformation model is not to be generated, the process shown in the flowchart in Figure 2 is terminated.
[0028] Here, we have described an example in which the operation feasibility determination unit 11 and the error determination unit 13 are executed each time a conversion model is generated, but this is not the only example. For example, multiple conversion models may be generated, and then the operation feasibility determination unit 11 and the error determination unit 13 may be executed for each conversion model.
[0029] With the model generation device 10 of this embodiment configured as described above, only the transformation models that are determined to be operable on the edge device by the execution of the inference runtime are selected as candidate models for machine learning from among the multiple transformation models generated by the model generation unit 1. This makes it possible to focus on transformation models that actually work on the edge device as the target for machine learning, and to develop models applicable to edge devices more efficiently.
[0030] Furthermore, according to the model generation device 10 of this embodiment, among the conversion models determined to be operable on the edge device, only conversion models in which the error between the execution result of the inference runtime by the conversion model and the execution result of the inference runtime by the original model is below a threshold are selected as candidate models for machine learning. Therefore, it is possible to narrow down the target of machine learning to conversion models that actually operate on the edge device and whose performance is guaranteed to a certain extent.
[0031] Next, we will describe an example of an application. Figure 3 is a block diagram showing an example of the functional configuration of a model generation device 20 related to an application. As shown in Figure 3, the model generation device 20 related to an application includes an original model generation unit 21, a constraint determination unit 22, a format conversion unit 1A, a model verification unit 2A, an optimization / simplification unit 1B, a model verification unit 2B, a graph format conversion unit 1C, a model verification unit 2C, a deployment conversion unit 1D, and a model verification unit 2D.
[0032] The above functional blocks 21, 22, 1A-1D, and 2A-2D can be configured using either hardware or software. For example, the above functional blocks 21, 22, 1A-1D, and 2A-2D are realized by the operation of programs stored in storage media such as RAM, ROM, hard disk, or semiconductor memory, controlled by a microcomputer configured with a CPU, RAM, ROM, etc. A GPU, FPGA, ASIC, or DSP may be used instead of or in addition to the CPU.
[0033] The format conversion unit 1A, the optimization / simplification unit 1B, the graph format conversion unit 1C, and the deployment conversion unit 1D are one embodiment of the model generation unit 1 shown in Figure 1. These conversion units 1A to 1D sequentially generate multiple patterns of converted models by converting the model components of the original model generated by the original model generation unit 21. Specifically, the format conversion unit 1A converts the format of the original model to generate a converted model. The optimization / simplification unit 1B converts the converted model generated by the format conversion unit 1A to generate another converted model. The graph format conversion unit 1C converts the converted model generated by the optimization / simplification unit 1B to generate another converted model. The deployment conversion unit 1D converts the converted model generated by the graph format conversion unit 1C to generate another converted model.
[0034] Model verification units 2A to 2D are all configured in the same way as model verification unit 2 shown in Figure 1. In model verification units 2A to 2D, the inference runtime executed by the operation feasibility determination unit 11 and the original model inference execution unit 12 is an inference runtime that is compatible with the conversion model generated by the format conversion unit 1A, optimization / simplification unit 1B, graph format conversion unit 1C, and deployment conversion unit 1D, which are located in the preceding stages. For example, the inference runtime executed in model verification unit 2A is an inference runtime that is compatible with the conversion model generated by the format conversion unit 1A.
[0035] In the model verification units 2A to 2D, the original models that the original model inference execution unit 12 targets for execution of the inference runtime are one or more original models that the constraint determination unit 22 has determined to satisfy predetermined constraints.
[0036] In the model verification units 2A to 2D, the threshold value set in the error determination unit 13 is set to an appropriate value according to the converted model generated by the preceding stages: the format conversion unit 1A, the optimization / simplification unit 1B, the graph format conversion unit 1C, and the deployment conversion unit 1D. For example, in the format conversion unit 1A, only the graph format of the original model is converted, so the threshold value of the error determination unit 13 in the model verification unit 2A is set to a small value. On the other hand, in the optimization / simplification unit 1B, the graph format conversion unit 1C, and the deployment conversion unit 1D, model components other than the graph format are also converted, so the threshold value of the error determination unit 13 in the model verification units 2B to 2D is set to a larger value than that of the model verification unit 2A.
[0037] In this way, the model generation device 20 in the application example sequentially generates conversion models of different patterns using a format conversion unit 1A, an optimization / simplification unit 1B, a graph format conversion unit 1C, and a deployment conversion unit 1D, each of which functions as an embodiment of the model generation unit 1. Each time a conversion model of a different pattern is generated, the model verification units 2A to 2D execute the operations feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13.
[0038] The original model generation unit 21 generates multiple patterns of original models in a specific graph format, with various changes to model components other than the graph format. The specific graph format is arbitrary, but for example, it may be the PyTorch graph format. That is, the original model generation unit 21 generates multiple patterns of original models in the PyTorch graph format, changing at least one of the following: the network hierarchy structure, the channel order format, and the numerical format.
[0039] The constraint determination unit 22 determines whether the multiple patterns of original models generated by the original model generation unit 21 satisfy predetermined device constraints, and discards original models that do not satisfy the device constraints. For example, the constraint determination unit 22 determines whether the generated original model is small enough to fit into the memory of the GPU used for machine learning. That is, it determines whether machine learning of the original model can be performed on the GPU being used. Then, it discards original models that do not satisfy the GPU memory constraints.
[0040] Furthermore, the constraint determination unit 22 may determine whether the parameters (weights) of the generated original model and the results of the convolution operation are within a predetermined upper limit, and discard original models that are not within the upper limit. Here, the upper limits for parameters and the upper limits for the convolution operation results can be set arbitrarily. For example, the upper limit for parameters can be set to the maximum number of weights calculated to fit into the memory or storage of the target edge device. Also, if the specifications of the edge device limit the number of convolution operations that can be processed in one frame, the upper limit for the convolution operation results can be set to that limit.
[0041] The format conversion unit 1A converts one or more original models determined by the constraint determination unit 22 to satisfy predetermined device constraints into models of multiple versions of a predetermined format. For example, the format conversion unit 1A converts an original model into multiple versions of ONNX models (hereinafter referred to as ONNX models). If there are multiple original models determined to satisfy predetermined device constraints, the format conversion unit 1A converts each of those multiple original models into multiple versions of ONNX models. Note that ONNX, as mentioned here, is just one example of a format and is not limited to it.
[0042] The model verification unit 2A executes the operations of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for multiple ONNX models generated by the format conversion unit 1A. The model verification unit 2A selects an ONNX model that is operable on an edge device and in which the error between the response information output from the inference runtime executed for the ONNX model and the response information output from the inference runtime executed for the original model is below a threshold, and notifies the optimization / simplification unit 1B.
[0043] The optimization / simplification unit 1B performs at least one of the optimization and / or simplification processes on one or more ONNX models selected by the model verification unit 2A from among multiple versions of ONNX models generated by the format conversion unit 1A. If both optimization and simplification processes are performed, for example, the optimization process is performed first, followed by the simplification process.
[0044] For example, the optimization / simplification unit 1B optimizes the ONNX model using an optimization tool called ONNX Optimizer, which is provided for the ONNX model. However, it is not always possible to use all the functions of ONNX Optimizer. Therefore, it is also possible to apply the functions of ONNX Optimizer to the model one by one and select and use only those that can be used without problems.
[0045] Furthermore, the optimization / simplification unit 1B simplifies the ONNX model using a simplification tool called ONNX Simplifier, which is provided for the ONNX model.
[0046] The model verification unit 2B executes the operations of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for one or more ONNX models that have been optimized and / or simplified by the optimization / simplification unit 1B. The model verification unit 2B selects an ONNX model that is operable on an edge device and in which the error between the response information output from the inference runtime executed for the ONNX model and the response information output from the inference runtime executed for the original model is below a threshold, and notifies the graph format conversion unit 1C of this selection.
[0047] The graph format conversion unit 1C converts one or more ONNX models selected by the model validation unit 2B from among the ONNX models generated by the optimization / simplification unit 1B into a model in a predetermined graph format. The graph format to be converted from the ONNX model is at least one of the graph formats of various frameworks such as Tensorflow, TFLite, Keras, OpenVINO, and scikit-learn. Note that the graph formats listed here are just examples and are not limited to them. Here, as an example, we will convert to models in three graph formats: Tensorflow, TFLite, and Keras. Hereafter, these converted models will be referred to as the Tensorflow model, the TFLite model, and the Keras model.
[0048] Here, the graph format conversion unit 1C converts the ONNX model into a Tensorflow model, a TFLite model, and a Keras model, respectively. Alternatively, the ONNX model may be converted into a Tensorflow model and a Keras model, respectively, and then the converted Tensorflow model and Keras model may be further converted into a TFLite model. Such combinations of conversions can be set arbitrarily.
[0049] The model verification unit 2C executes the operations of the feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for each of the multiple conversion models (e.g., Tensorflow model, TFLite model, Keras model) generated by the graph format conversion unit 1C. Specifically, it executes the operations of the model verification unit 2C for the Tensorflow model, the operations of the model verification unit 2C for the TFLite model, and the operations of the model verification unit 2C for the Keras model.
[0050] The model verification unit 2C then selects a conversion model that is operable on an edge device and in which the error between the response information output from the inference runtime executed for each conversion model and the response information output from the inference runtime executed for the original model is below a threshold, and notifies the deployment conversion unit 1D of this selection.
[0051] The deployment conversion unit 1D converts one or more conversion models selected by the model verification unit 2C from among the conversion models generated by the graph format conversion unit 1C into a model format for deployment to the target edge device. This conversion is performed using a dedicated conversion tool provided by the edge device manufacturer to convert the model into one that satisfies the hardware limitations of the edge device. Hardware limitations of the device include, for example, available computational operations, memory capacity, storage capacity, computational power that can be processed at one time, power consumption, and inference speed. Here, computational operations refer to the addition of convolutional layers, pooling layers, Transformer layers, Attention layers, normalization layers, MLP layers, and fully connected layers, as well as the activation functions used. Note that the hardware limitations and computational operations of the device listed here are examples and are not limited to them.
[0052] The model verification unit 2D executes the operations of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 on the transformed model generated by the deployment transformation unit 1D. The model verification unit 2D then selects a transformed model that is operable on an edge device and in which the error between the response information output from the inference runtime executed for the transformed model and the response information output from the inference runtime executed for the original model is below a threshold, and notifies the deployment transformation unit 1D. The deployment transformation unit 1D adopts the model selected by the model verification unit 2D as a candidate model for machine learning.
[0053] In the model generation device 20 configured as described above, different pattern conversion models are generated sequentially, and each time a different pattern conversion model is generated, a determination of whether it can operate on the edge device and an error determination with the original model are performed. This makes it possible to efficiently narrow down and select a conversion model that can operate on the edge device and guarantees a certain level of performance from among many pattern conversion models. Furthermore, since the threshold values for the error determination performed by each model verification unit 2A to 2D are changed to match the conversion model generated by each conversion unit 1A to 1D, quality assurance tailored to the conversion model can be achieved.
[0054] In the model generation device 20 shown in Figure 3, an example was described in which the model generation unit 1 includes a format conversion unit 1A, an optimization / simplification unit 1B, a graph format conversion unit 1C, and a deployment conversion unit 1D. However, it is not necessary to include all of these. For example, the processing of the optimization / simplification unit 1B may be omitted for some or all of the graph formats converted by the graph format conversion unit 1C. In this case, the graph format conversion unit 1C converts the conversion model generated by the format conversion unit 1A and selected by the model verification unit 2A into a model in a predetermined graph format. Alternatively, the processing of the format conversion unit 1A and the optimization / simplification unit 1B may be omitted for some or all of the graph formats converted by the graph format conversion unit 1C. In this case, the graph format conversion unit 1C converts the original model generated by the original model generation unit 21 and selected by the constraint determination unit 22 into a model in a predetermined graph format.
[0055] Furthermore, in the above embodiment, an example was described in which the model verification unit 2 (2A to 2D) performs both a determination of whether the model can operate and a determination of the error in the model output. However, it is also possible to perform only the determination of whether the model can operate. In this case, the configuration of the original model inference execution unit 12 and the error determination unit 13 is omitted, and the model verification unit 2 (2A to 2D) will only perform the processing of the operation feasibility determination unit 11.
[0056] While this configuration may result in a large number of transformation models being selected for machine learning, it is possible to narrow down the selection to only those transformation models that can operate on edge devices. The prediction accuracy of the models generated by machine learning will be evaluated, thereby ensuring a certain level of model quality. However, it is preferable to narrow down the number of transformation models to be trained by checking the error of the model output before performing time-consuming machine learning, and this will significantly improve development efficiency.
[0057] Furthermore, while the above embodiment describes an example in which a conversion model determined to be inoperable by the operation feasibility determination unit 11 and a conversion model determined to have an error greater than a threshold by the error determination unit 13 are discarded, the present invention is not limited thereto. For example, as shown in Figure 4, when a conversion model is determined to be inoperable by the operation feasibility determination unit 11 and when an error determined to have an error greater than a threshold by the error determination unit 13, the conversion model itself is discarded, while the information of that conversion model is fed back to the model generation unit 1, and the model generation unit 1 modifies the model components based on the feedback information and regenerates a different conversion model. Note that the feedback of the determination result may be performed by only one of the operation feasibility determination unit 11 or the error determination unit 13.
[0058] Furthermore, the above embodiments are merely examples of how the present invention may be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its gist or its main features. [Explanation of Symbols]
[0059] 1. Model Generation Unit 1A Format conversion unit 1B Optimization / Simplified Section 1C Graph format conversion unit 1D Transformation Unit for Deployment 2,2A~2D Model Verification Section 10 Model Generators 11 Operation possibility determination section 12. Original Model Inference Execution Unit 13 Error judgment section 20 Model Generators 21 Original Model Generation Unit 22 Constraint judgment part
Claims
1. A model generation unit that generates multiple model patterns by transforming a model component of an original model, which includes at least one of the network hierarchy, graph format, channel order format, and numerical format, The above-mentioned model generation unit executes an inference runtime on multiple model patterns to determine whether they can operate on edge devices, and the operation feasibility determination unit selects the models that can operate as candidate models for machine learning. The original model inference execution unit executes the above inference runtime for the original model described above, The system includes an error determination unit that determines whether the error between the response information output from the inference runtime for models determined to be operable by the above-mentioned operable status determination unit and the response information output from the inference runtime for the original model is below a threshold, and selects models from among the operable models whose error is below the threshold as candidate models for machine learning. A model generation device characterized by the following features.
2. The above model generation unit includes a format conversion unit that converts the above original model into multiple versions of a model in a predetermined format. The above-mentioned operation feasibility determination unit and error determination unit process the multiple models generated by the above-mentioned format conversion unit. The model generation apparatus according to feature 1.
3. The above model generation unit further includes an optimization / simplification unit that performs at least one of optimization and simplification on multiple versions of the model generated by the above format conversion unit. The above-mentioned operation feasibility determination unit and error determination unit process the multiple models generated by the above-mentioned optimization / simplification unit. The model generation apparatus according to feature 2.
4. The above model generation unit includes a graph format conversion unit that converts the above original model, the model generated by the above format conversion unit, or the model generated by the above optimization / simplification unit into a model in a predetermined graph format. The above-mentioned operational feasibility determination unit and the above-mentioned error determination unit are executed on the multiple models generated by the above-mentioned graph format conversion unit. The model generation apparatus according to feature 3.
5. The above model generation unit includes a deployment conversion unit that converts the model generated by the above graph format conversion unit into a model in a format suitable for deployment to the edge device. The above-mentioned operational feasibility determination unit and the above-mentioned error determination unit process the model generated by the above-mentioned deployment conversion unit. The model generation apparatus according to feature 4.
6. The model generation apparatus according to any one of claims 1 to 5, characterized in that the model generation unit regenerates another model by changing the model components based on model information in at least one of the cases in which the operation feasibility determination unit determines that the edge device is inoperable and the error determination unit determines that the error is greater than the threshold.
7. The above model generation unit sequentially generates models of different patterns. Each time the above-mentioned model generation unit generates a model with a different pattern, the above-mentioned operation feasibility determination unit and the above-mentioned error determination unit execute their respective processes. A model generation apparatus according to any one of claims 1 to 5.
8. The model generation apparatus according to claim 7, characterized in that, with respect to the processing of the error determination unit which is executed each time the above-mentioned model generation unit generates a model of a different pattern, the threshold value is changed according to the model generated.
9. The computer's model generation unit generates multiple model patterns by transforming model components of the original model, including at least one of the network hierarchy, graph format, channel order format, and numerical format. The above-mentioned computer's operational feasibility determination unit performs an inference runtime on multiple model patterns generated by the above-mentioned model generation unit to determine whether they can be operated on an edge device, and selects the operational models as candidate models for machine learning. The original model inference execution unit of the above computer executes the inference runtime for the above original model, The error determination unit of the above computer has the following steps: it determines whether the error between the response information output from the inference runtime for the model determined to be operable by the operableness determination unit and the response information output from the inference runtime for the original model is below a threshold, and selects from the models determined to be operable that have an error below the threshold as candidate models for machine learning. A model generation method characterized by the following: