Model generation device and model generation method

Abstract

To provide a model generation device and a model generation method that can more efficiently develop a model applicable to an edge device by limiting machine learning targets to a model that actually operates on the edge device.SOLUTION: A model generation device includes: a model generation unit 1 that generates models with a plurality of patterns by changing a model component including at least one of a hierarchical structure of a network, a graph format, a channel order format, and a numerical format; and a model verification unit 2 that executes inference runtime for the generated models with the plurality of patterns to determine whether they can be operated on the edge device and selects a model that can be operated as candidate models for machine learning. The model generation device narrows down the generated models to only those determined to be operable on the edge device, and uses them as targets for machine learning, thereby enabling more efficient development of a model applicable to the edge device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a model generation apparatus and a model generation method, and more particularly to an apparatus and a method for generating a model to be installed in an edge device. [Background technology]

[0002] In recent years, the use of AI (artificial intelligence) using machine learning has become more widespread. Technology is also being developed that combines AI-related technology with IoT (Internet of Things)-related technology to control the operation of various devices such as home appliances and industrial equipment based on the results of inferences made using models created through machine learning. Edge devices are terminal devices connected to a network, and by equipping these edge devices with AI, it becomes possible to make instantaneous decisions and control the devices at the site where the data is generated.

[0003] In general, to create a model by performing machine learning, it is necessary to collect a large amount of learning data. In addition, to create an appropriate model that matches the purpose, it is necessary to improve the model through many trials. In general, a huge number of trial and error processes are required before an optimal model for an edge device is obtained, and a lot of development time is required. In order to address this problem, a technology has been known that narrows down candidates for neural networks to be used in machine learning and makes it possible to efficiently obtain an optimal model (see, for example, Patent Document 1).

[0004] The neural network construction device described in Patent Document 1 determines candidate hyperparameters for the neural network using a first condition related to the computational resources of the embedded device (conditions of the execution environment related to the hardware such as the CPU operating frequency and memory size, the size of the input data to the model and the size of the output data from the model, the target accuracy of inference by the model, etc.), and generates a neural network model using optimal candidate hyperparameters searched from a candidate list using Bayesian optimization, etc. Then, for the generated model, it is determined whether or not it satisfies a second condition related to the performance that the model should have (performance constraints such as a target processing time), and models that do not satisfy the performance constraints are discarded, while machine learning is performed using the learning data for models that satisfy the performance constraints, the predictive accuracy of the models is evaluated, and then the models are added to a list of evaluated models.

[0005] According to the description of Patent Document 1, by generating a neural network that satisfies the first condition, it is possible to efficiently narrow down the candidates for the optimal neural network to neural networks that have a configuration that satisfies the resource condition determined according to the problem to be solved. In addition, by adopting only the models that satisfy the second condition from among the narrowed down models as the learning target models, it is possible to screen out models that do not have the performance required according to the application, even if they satisfy the scale constraint, and narrow down the models to be used for machine learning and prediction accuracy evaluation. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6799197 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the technology described in Patent Document 1, although candidate hyperparameters of a neural network are determined so as to satisfy the first condition, it is not confirmed whether a model generated using the candidate hyperparameters actually runs on an edge device. Therefore, a model that does not actually run on an edge device may be subjected to an evaluation based on the second condition and subjected to machine learning and prediction accuracy evaluation, resulting in a problem of performing unnecessary processing.

[0008] The present invention has been made to solve such problems, and aims to enable more efficient development of models that can be applied to edge devices by narrowing down the targets of machine learning to models that actually run on edge devices. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, the present invention generates multiple patterns of models by changing model components including at least one of the hierarchical structure of the network, the graph format, the channel order format, and the numerical format, executes an inference runtime for the generated multiple patterns of models to determine whether they can operate on an edge device, and selects models that can operate as candidate models for machine learning. Effect of the Invention

[0010] According to the present invention configured as described above, from among the multiple generated model patterns, only models that are determined to be operable on an edge device are selected as candidate models for machine learning. This makes it possible to narrow down the models that can actually operate on an edge device as targets for machine learning, thereby making it possible to more efficiently develop models that can be applied to edge devices. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a basic functional configuration of a model generating device according to an embodiment of the present invention; [Diagram 2] 4 is a flowchart showing an example of the operation of the model generating device according to the present embodiment. [Diagram 3] FIG. 13 is a block diagram showing an example of a functional configuration of a model generating device according to an application example. [Figure 4] FIG. 13 is a block diagram showing another example of the functional configuration of the model generating device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a block diagram showing an example of a basic functional configuration of a model generation device 10 according to this embodiment. As shown in Fig. 1, the model generation device 10 of this embodiment includes, as functional configurations, a model generation unit 1 and a model verification unit 2. The model verification unit 2 includes, as specific functional configurations, an operation feasibility determination unit 11, an original model inference execution unit 12, and an error determination unit 13.

[0013] The functional blocks 1 and 2 can be configured by either hardware or software. For example, the functional blocks 1 and 2 are realized by the operation of a program stored in a storage medium such as a RAM, a ROM, a hard disk, or a semiconductor memory under the control of a microcomputer configured with a CPU, a RAM, a ROM, etc. Instead of or in addition to the CPU, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), etc. may be used.

[0014] The model generation unit 1 generates a plurality of patterns of models by changing the model components. The model components include at least one of a hierarchical structure of a network, a graph format, a channel order format, and a numerical format.

[0015] The hierarchical structure of a network is a structure related to the combination, order, size, etc. of the layers that make up the model, and includes at least one of the following structures: convolutional layer, pooling layer, transformer layer, attention layer, normalization layer, MLP layer, fully connected layer, layer composite block layer composed of these layers, activation function, how layers are connected or aggregation layer, backbone configuration, neck configuration (e.g. deconvolution layer, upsample layer, feature pyramid layer, etc.), head configuration.

[0016] The graph format is the format of the model's computation graph, and includes various frameworks such as ONNX, PyTorch, Tensorflow, TFLite, Keras, OpenVINO, and scikit-learn. The channel order format is the channel order format used in the model's calculations, and for example, if the data handled by the model is images, the format includes information such as the number of images, the number of channels (color, etc.), and the height and width of the images. The number format is the format of the numbers used for the model's weights or calculations, and includes information such as fp32, fp16, bfloat16, and int8.

[0017] The model generation unit 1 generates a plurality of patterns of models (hereinafter, referred to as conversion models) by converting model components of an original model generated by, for example, combining predetermined model components. The original model may be generated in advance by a device other than the model generation device 10, or may be generated by the model generation unit 1.

[0018] The model verification unit 2 verifies whether each of the multiple patterns of converted models generated by the model generation unit 1 can be operated on the edge device and whether the performance error with the original model is within a certain range. Then, only the converted models that are confirmed to be operable on the edge device and have a performance error with the original model within a certain range are selected as candidate models for machine learning.

[0019] The operation feasibility determination unit 11 of the model verification unit 2 executes an inference runtime for the multiple patterns of conversion models generated by the model generation unit 1 to determine whether they can be operated on the edge device, and selects an operable model as a candidate model for machine learning. Strictly speaking, an operable model is selected as a candidate model for determination by the error determination unit 13. Types of inference runtime include software that has both the functions of developing and executing models, from which the development function is omitted and only the execution function is extracted, as well as an inference runtime based on a device emulator and an inference runtime that deploys and executes a model on evaluation hardware.

[0020] The inference runtime is prepared for each of various frameworks and edge device types. The operation feasibility determination unit 11 uses one of the multiple types of inference runtimes that is compatible with the conversion model, executes inference using verification data prepared in advance, and verifies whether the conversion model is operable. Here, the operation feasibility determination unit 11 determines that the conversion model is inoperable when no output is obtained from the inference runtime. In addition, when the inference runtime has a check function, the operation feasibility determination unit 11 can also determine whether the conversion model is operable based on whether an error is output by the check function. In addition, the operation feasibility determination unit 11 determines that the conversion model is operable when a normal output, not an error, is obtained from the inference runtime. Note that, when the format of the channel order or the format of the numerical value has changed due to the conversion process from the original model to the conversion model, the verification data to be input to the conversion model is adjusted so as to match the format of the data to be input to the conversion model.

[0021] In addition, if there is no inference runtime that is compatible with the conversion model, an attempt may be made to convert the model using a dedicated conversion tool provided by the edge device manufacturer, and if the conversion is successful, it may be determined that the model is operational, whereas if the conversion fails, it may be determined that the model is inoperable.

[0022] The original model inference execution unit 12 executes an inference runtime for the original model used for conversion by the model generation unit 1. Here, the original model inference execution unit 12 executes inference using an inference runtime that is compatible with the original model, and the same verification data as the verification data used by the operation feasibility determination unit 11. If the channel order format or the numeric format has changed due to the conversion process from the original model to the converted model, the output data from the inference runtime used for inference of the original model is adjusted so as to match the format of the data output by the converted model.

[0023] The error judgment unit 13 judges whether the error (e.g., mean squared error) between the response information output from the inference runtime executed for the converted model judged to be operable by the operation feasibility judgment unit 11 and the response information output from the inference runtime executed for the original model by the original model inference execution unit 12 is below a threshold, and selects, from among the converted models judged to be operable, a converted model whose error is below the threshold as a candidate model for machine learning.

[0024] FIG. 2 is a flowchart showing an example of the operation of the model generating device 10 according to this embodiment configured as described above.

[0025] First, the model generation unit 1 generates a converted model by converting the model components of the original model (step S1). Next, the operation feasibility determination unit 11 of the model verification unit 2 executes an inference runtime for the converted model generated by the model generation unit 1 (step S2) and determines whether it can operate on the edge device (step S3). If it is determined that the converted model is inoperable, the converted model generated in step S1 is discarded (step S7), and the process proceeds to step S8. Discarding the converted model means excluding it from the machine learning processing targets.

[0026] On the other hand, if it is determined that the conversion model is operable, the original model inference execution unit 12 executes inference using an inference runtime that matches the original model and the same verification data as that used by the operation feasibility determination unit 11 (step S4).Then, the error determination unit 13 determines whether or not the error between the response information output from the inference runtime executed for the converted model and the response information output from the inference runtime executed for the original model is equal to or smaller than a threshold value (step S5).

[0027] Here, if it is determined that the error is not equal to or less than the threshold, the conversion model generated in step S1 is discarded (step S7), and the process proceeds to step S8. On the other hand, if it is determined that the error is equal to or less than the threshold, the error determination unit 13 selects the conversion model generated in step S1 as a candidate model for machine learning (step S6). Thereafter, the process proceeds to step S8. In step S8, it is determined whether or not to end the process. Here, it is determined whether or not to generate another conversion model. If another conversion model is to be generated, the process returns to step S1. On the other hand, if another conversion model is not to be generated, the process of the flowchart shown in FIG. 2 is ended.

[0028] Although an example in which the operation feasibility determination unit 11 and the error determination unit 13 are executed every time one conversion model is generated has been described here, the present invention is not limited to this. For example, after multiple conversion models are generated, the operation feasibility determination unit 11 and the error determination unit 13 may be executed for each conversion model.

[0029] According to the model generation device 10 of this embodiment configured as described above, among the multiple patterns of conversion models generated by the model generation unit 1, only those conversion models that are determined to be operable on an edge device by execution of the inference runtime are selected as candidate models for machine learning. This makes it possible to narrow down the conversion models that can actually operate on the edge device as targets for machine learning, thereby making it possible to more efficiently develop models that can be applied to edge devices.

[0030] Furthermore, according to the model generation device 10 of this embodiment, among the conversion models determined to be operable on an edge device, only those conversion models for which the error between the execution result of the inference runtime using the conversion model and the execution result of the inference runtime using the original model is below a threshold are selected as candidate models for machine learning, so that it is possible to narrow down the conversion models to be targets for machine learning to those that actually operate on an edge device and have a certain level of performance guaranteed.

[0031] Next, an application example will be described. Fig. 3 is a block diagram showing an example of a functional configuration of a model generation device 20 according to the application example. As shown in Fig. 3, the model generation device 20 according to the application example includes an original model generation unit 21, a constraint determination unit 22, a format conversion unit 1A, a model verification unit 2A, an optimization / simplification unit 1B, a model verification unit 2B, a graph format conversion unit 1C, a model verification unit 2C, a deployment conversion unit 1D, and a model verification unit 2D.

[0032] The above-mentioned functional blocks 21, 22, 1A to 1D, 2A to 2D can be configured by either hardware or software. For example, the above-mentioned functional blocks 21, 22, 1A to 1D, 2A to 2D are realized by the operation of a program stored in a storage medium such as a RAM, a ROM, a hard disk, or a semiconductor memory under the control of a microcomputer configured with a CPU, a RAM, a ROM, etc. Instead of or in addition to the CPU, a GPU, an FPGA, an ASIC, a DSP, etc. may be used.

[0033] The format conversion unit 1A, the optimization / simplification unit 1B, the graph format conversion unit 1C, and the deployment conversion unit 1D are one aspect of the model generation unit 1 shown in FIG. 1. These conversion units 1A to 1D convert model components of an original model generated by the original model generation unit 21 to sequentially generate a plurality of patterns of conversion models. That is, the format conversion unit 1A converts the format of the original model to generate a conversion model. The optimization / simplification unit 1B converts the conversion model generated by the format conversion unit 1A to generate another conversion model. The graph format conversion unit 1C converts the conversion model generated by the optimization / simplification unit 1B to generate another conversion model. The deployment conversion unit 1D converts the conversion model generated by the graph format conversion unit 1C to generate another conversion model.

[0034] The model verification units 2A to 2D are each configured in the same manner as the model verification unit 2 shown in Fig. 1. In the model verification units 2A to 2D, the inference runtime executed by the operation feasibility determination unit 11 and the original model inference execution unit 12 is an inference runtime adapted to the conversion model generated by the format conversion unit 1A, the optimization / simplification unit 1B, the graph format conversion unit 1C, and the conversion unit for deployment 1D, which are located in the preceding stages, respectively. For example, the inference runtime executed in the model verification unit 2A is an inference runtime adapted to the conversion model generated by the format conversion unit 1A.

[0035] In the model verification units 2A to 2D, the original models that the original model inference execution unit 12 executes the inference runtime on are one or more original models that the constraint determination unit 22 determines as satisfying predetermined constraints.

[0036] In the model verification units 2A to 2D, the threshold value set in the error determination unit 13 is set to an appropriate value in accordance with the converted model generated by the format conversion unit 1A, optimization / simplification unit 1B, graph format conversion unit 1C, and conversion unit for deployment 1D, which are located in the preceding stages. For example, since the format conversion unit 1A only converts the graph format of the original model, the threshold value of the error determination unit 13 in the model verification unit 2A is set to a small value. On the other hand, since the optimization / simplification unit 1B, graph format conversion unit 1C, and conversion unit for deployment 1D also convert model components other than the graph format, the threshold value of the error determination unit 13 in the model verification units 2B to 2D is set to a larger value than that of the model verification unit 2A.

[0037] In this manner, in the model generating device 20 according to the application example, conversion models of different patterns are sequentially generated by the format conversion unit 1A, optimization / simplification unit 1B, graph format conversion unit 1C, and conversion unit for deployment 1D, each of which functions as one aspect of the model generating unit 1. Then, each time a conversion model of a different pattern is generated, the model verification units 2A to 2D execute the processes of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13.

[0038] The original model generation unit 21 generates a plurality of patterns of original models in a specific graph format by changing model components other than the graph format in various ways. The specific graph format is arbitrary, but may be, for example, the graph format of PyTorch. That is, the original model generation unit 21 generates a plurality of patterns of original models in the graph format of PyTorch by changing at least one of the hierarchical structure of the network, the format of the channel order, and the format of the numerical values.

[0039] The constraint determination unit 22 determines whether a plurality of patterns of original models generated by the original model generation unit 21 satisfy a predetermined device constraint, and discards original models that do not satisfy the device constraint. For example, the constraint determination unit 22 determines whether the generated original model has a size that can be stored in the memory of the GPU used for machine learning. In other words, it determines whether the machine learning of the original model can be executed on the GPU being used. Then, it discards original models that do not satisfy the constraint of the GPU memory.

[0040] Furthermore, the constraint determination unit 22 may determine whether the parameters (weights) of the generated original model and the results of the convolution operation are within a preset upper limit, and discard the original model that is not within the upper limit. Here, the upper limit of the parameters and the upper limit of the convolution operation result may be set arbitrarily. For example, the upper limit of the parameters may be set to the maximum number of weights calculated so as to fit into the memory or storage of the target edge device. Furthermore, the upper limit of the convolution operation result may be set to the limited value when the specifications of the edge device limit the convolution operation that can be processed within one frame.

[0041] The format conversion unit 1A converts one or more original models determined by the constraint determination unit 22 to satisfy a predetermined device constraint into a plurality of versions of models in a predetermined format. For example, the format conversion unit 1A converts the original models into a plurality of versions of ONNX models (hereinafter, referred to as ONNX models). When there are a plurality of original models determined to satisfy a predetermined device constraint, the format conversion unit 1A converts the plurality of original models into a plurality of versions of ONNX models. Note that ONNX listed here is an example of a format, and is not limited thereto.

[0042] The model verification unit 2A executes the processes of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for the multiple ONNX models generated by the format conversion unit 1A. The model verification unit 2A selects an ONNX model that is operable on the edge device and for which the error between response information output from the inference runtime executed for the ONNX model and response information output from the inference runtime executed for the original model is equal to or less than a threshold, and notifies the optimization / simplification unit 1B.

[0043] The optimization / simplification unit 1B performs at least one of optimization and simplification processing on one or more ONNX models selected by the model verification unit 2A from among the multiple versions of ONNX models generated by the format conversion unit 1A. When performing both optimization and simplification processing, for example, the optimization processing is performed first, and then the simplification processing is performed.

[0044] For example, the optimization / simplification unit 1B optimizes the ONNX model using an optimization tool called ONNX Optimizer that is provided for the ONNX model. Here, it is not always possible to use all of the functions of the ONNX Optimizer. Therefore, it is possible to apply the functions of the ONNX Optimizer one by one to the model and select and use those that can be used without problems.

[0045] In addition, the optimization / simplification unit 1B simplifies the ONNX model using a simplification tool called ONNX Simplifier, which is provided for the ONNX model.

[0046] The model verification unit 2B executes the processes of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for one or more ONNX models optimized and / or simplified by the optimization / simplification unit 1B. The model verification unit 2B selects an ONNX model that is operable on the edge device and for which the error between response information output from the inference runtime executed for the ONNX model and response information output from the inference runtime executed for the original model is equal to or less than a threshold, and notifies the graph format conversion unit 1C.

[0047] The graph format conversion unit 1C converts one or more ONNX models selected by the model verification unit 2B from the ONNX models generated by the optimization / simplification unit 1B into a model in a predetermined graph format. The graph format to which the ONNX model is converted is at least one of the graph formats of various frameworks such as Tensorflow, TFLite, Keras, OpenVINO, scikit-learn, etc. Note that the graph formats listed here are examples and are not limited to these. As an example, the models are converted into three graph formats, Tensorflow, TFLite, and Keras. Hereinafter, these converted models are referred to as Tensorflow model, TFLite model, and Keras model.

[0048] Here, the graph format conversion unit 1C converts the ONNX model into a Tensorflow model, a TFLite model, and a Keras model, respectively. Alternatively, the ONNX model may be converted into a Tensorflow model and a Keras model, respectively, and the converted Tensorflow model and Keras model may be further converted into a TFLite model. Such a combination of conversions can be set arbitrarily.

[0049] The model verification unit 2C executes the processes of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for each of the multiple conversion models (e.g., Tensorflow model, TFLite model, Keras model) generated by the graph format conversion unit 1C. That is, the model verification unit 2C executes the processes of the model verification unit 2C for the Tensorflow model, the model verification unit 2C executes the processes of the model verification unit 2C for the TFLite model, and the model verification unit 2C executes the processes of the model verification unit 2C for the Keras model.

[0050] Then, the model verification unit 2C selects a conversion model that can be operated on an edge device and in which the error between the response information output from the inference runtime executed for each conversion model and the response information output from the inference runtime executed for the original model is below a threshold value, and notifies the deployment conversion unit 1D.

[0051] The conversion unit for deployment 1D converts one or more conversion models selected by the model verification unit 2C from among the conversion models generated by the graph format conversion unit 1C into a model in a format for deployment to a target edge device. This conversion is performed using a dedicated conversion tool provided by the edge device manufacturer to convert into a model that satisfies the hardware limitations of the edge device. The hardware limitations of the device include, for example, available calculation operations, memory capacity, storage capacity, computing power that can be processed at one time, power consumption, inference speed, etc. Here, the calculation operations refer to the addition of a convolution layer, pooling layer, transformer layer, attention layer, normalization layer, MLP layer, fully connected layer, and the activation function used. Note that the hardware limitations and calculation operations of the device listed here are merely examples and are not limited to these.

[0052] The model verification unit 2D executes the processes of the operation feasibility determination unit 11, the original model inference execution unit 12, and the error determination unit 13 for the converted model generated by the conversion unit for deployment 1D. Then, the model verification unit 2D selects a converted model that can operate on the edge device and has an error between response information output from an inference runtime executed for the converted model and response information output from an inference runtime executed for the original model that is equal to or smaller than a threshold, and notifies the conversion unit for deployment 1D. The conversion unit for deployment 1D employs the model selected by the model verification unit 2D as a candidate model for machine learning.

[0053] In the model generating device 20 according to the application example configured as described above, conversion models of different patterns are generated in sequence, and each time a conversion model of a different pattern is generated, a judgment as to whether it can be operated on an edge device and an error judgment with respect to the original model are performed, so that it is possible to efficiently narrow down and select a conversion model that can be operated on an edge device and has a certain level of performance from among many conversion models. In addition, the threshold value of the error judgment performed by each of the model verification units 2A to 2D is changed according to the conversion model generated by each of the conversion units 1A to 1D, so that it is possible to realize quality assurance according to the conversion model.

[0054] In the model generating device 20 shown in FIG. 3, an example has been described in which the format conversion unit 1A, the optimization / simplification unit 1B, the graph format conversion unit 1C, and the conversion unit for deployment 1D are provided as one aspect of the model generating unit 1, but it is not essential to provide all of these. For example, the processing of the optimization / simplification unit 1B may be omitted for some or all of the graph formats converted by the graph format conversion unit 1C. In this case, the graph format conversion unit 1C converts the conversion model generated by the format conversion unit 1A and selected by the model verification unit 2A into a model in a predetermined graph format. In addition, the processing of the format conversion unit 1A and the optimization / simplification unit 1B may be omitted for some or all of the graph formats converted by the graph format conversion unit 1C. In this case, the graph format conversion unit 1C converts the original model generated by the original model generating unit 21 and selected by the constraint determination unit 22 into a model in a predetermined graph format.

[0055] In the above embodiment, the model verification unit 2 (2A to 2D) judges whether or not the model is operable and judges the error of the model output. However, the model verification unit 2 (2A to 2D) may only judge whether or not the model is operable. In this case, the original model inference execution unit 12 and the error determination unit 13 are omitted, and the model verification unit 2 (2A to 2D) only executes the processing of the operation determination unit 11.

[0056] In this configuration, although the number of conversion models selected as targets for machine learning may be large, it is possible to narrow down the conversion models to those that can run on edge devices and perform machine learning. The prediction accuracy of the model generated by performing machine learning is evaluated, so it is possible to guarantee a certain level of quality for the model. However, it is more preferable to narrow down the number of conversion models to be learned by performing an error judgment on the model output before performing machine learning, which takes a lot of time, and this has a significant effect on improving development efficiency.

[0057] In the above embodiment, the example of discarding the conversion model determined to be inoperable by the operation possibility determination unit 11 and the conversion model determined to have an error larger than the threshold value by the error determination unit 13 has been described, but the present invention is not limited to this. For example, as shown in Fig. 4, when the operation possibility determination unit 11 determines that the conversion model is inoperable and when the error determination unit 13 determines that the error is larger than the threshold value, the conversion model determined in this way may be discarded, while information on the conversion model is fed back to the model generation unit 1, and the model generation unit 1 may change the model components based on the fed back information to regenerate another conversion model. Note that the feedback of the judgment result may be performed only from either the operation possibility determination unit 11 or the error determination unit 13.

[0058] In addition, the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited thereby. In other words, the present invention can be implemented in various forms without departing from the gist or main characteristics thereof. [Explanation of symbols]

[0059] 1. Model Generation 1A Format conversion section 1B Optimization / Simplification Section 1C Graph format conversion section Converter for 1D deployment 2,2A~2D Model Verification Section 10. Model Generator 11 Operation possibility determination section 12 Original model inference execution unit 13 Error judgment section 20 Model generation device 21 Original model generation section 22 Constraint judgment part

Claims

1. a model generating unit that generates a plurality of patterns of models by changing model components including at least one of a hierarchical structure of a network, a graph format, a channel order format, and a numerical format; and an operation feasibility determination unit that executes an inference runtime to determine whether or not the model can be operated on the edge device for the multiple patterns of models generated by the model generation unit, and selects an operable model as a candidate model for machine learning. A model generating device comprising:

2. The model generation unit includes a format conversion unit that converts an original model into a plurality of versions of a predetermined format; The operation determination unit executes a process for a plurality of models generated by the format conversion unit.

2. The model generating device according to claim 1.

3. the model generation unit further includes an optimization / simplification unit that performs at least one of optimization and simplification processing on the multiple versions of the models generated by the format conversion unit; The operation feasibility determination unit executes processing for a plurality of models generated by the optimization / simplification unit.

3. The model generating device according to claim 2.

4. the model generation unit includes a graph format conversion unit that converts the original model, the model generated by the format conversion unit, or the model generated by the optimization / simplification unit into a model in a predetermined graph format; The operation possibility determination unit executes a process for a plurality of models generated by the graph format conversion unit.

4. The model generating device according to claim 3.

5. the model generation unit includes a deployment conversion unit that converts the model generated by the graph format conversion unit into a model in a format for deployment to the edge device; The operation determination unit executes a process for the model generated by the conversion unit for deployment.

5. The model generating device according to claim 4.

6. The model generation device according to any one of claims 1 to 5, characterized in that the model generation unit regenerates another model by changing the model components based on model information regarding a case in which the edge device is determined to be inoperable by the operation feasibility determination unit.

7. the model generation unit generates the plurality of patterns of models by transforming the model components of an original model; an original model inference execution unit that executes the inference runtime on the original model; and an error determination unit that determines whether an error between response information output from the inference runtime for a model determined to be operable by the operation feasibility determination unit and response information output from the inference runtime for the original model is equal to or less than a threshold, and selects a model with the error equal to or less than the threshold from among the models determined to be operable as a candidate model for machine learning.

2. The model generating device according to claim 1.

8. the model generation unit includes a format conversion unit that converts the original model into a plurality of versions of a predetermined format; The operation feasibility determination unit and the error determination unit perform processing on a plurality of models generated by the format conversion unit.

8. The model generating device according to claim 7.

9. the model generation unit further includes an optimization / simplification unit that performs at least one of optimization and simplification processing on the multiple versions of the models generated by the format conversion unit; The operation feasibility determination unit and the error determination unit execute processing for a plurality of models generated by the optimization / simplification unit.

9. The model generating device according to claim 8.

10. the model generation unit includes a graph format conversion unit that converts the original model, the model generated by the format conversion unit, or the model generated by the optimization / simplification unit into a model in a predetermined graph format; The operation feasibility determination unit and the error determination unit perform processing on a plurality of models generated by the graph format conversion unit.

10. The model generating device according to claim 9.

11. the model generation unit includes a deployment conversion unit that converts the model generated by the graph format conversion unit into a model in a format for deployment to the edge device; The model generated by the conversion unit for deployment is subjected to processing by the operation feasibility determination unit and the error determination unit.

11. The model generating device according to claim 10.

12. The model generation device according to any one of claims 7 to 11, characterized in that the model generation unit regenerates another model by changing the model components based on model information relating to at least one of a case where the edge device is judged to be inoperable by the operation feasibility determination unit and a case where the error determination unit determines that the error is greater than the threshold value.

13. The model generation unit sequentially generates models of different patterns, Each time the model generating unit generates a model of a different pattern, the operation determination unit executes a process.

6. The model generating device according to claim 1, wherein the model generating device is a computer-readable storage device.

14. The model generation unit sequentially generates models of different patterns, Each time the model generating unit generates a model of a different pattern, the operation determination unit and the error determination unit execute processing. The model generating device according to any one of claims 7 to 11.

15. 15. The model generating device according to claim 14, wherein, in a process of the error determining unit which is executed each time a model of a different pattern is generated by the model generating unit, the threshold value is varied depending on the model to be generated.

16. A model generating unit of the computer generates a model of a plurality of patterns by changing model components including at least one of a hierarchical structure of a network, a graph format, a channel order format, and a numerical format; an operation feasibility determination unit of the computer executing an inference runtime for a plurality of patterns of models generated by the model generation unit to determine whether the models can be operated on the edge device, and selecting an operable model as a candidate model for machine learning; A model generation method comprising:

17. the model generation unit generates the plurality of patterns of models by transforming the model components of an original model; An original model inference execution unit of the computer executes the inference runtime on the original model; The method further includes a step in which an error determination unit of the computer determines whether an error between response information output from the inference runtime for a model determined to be operable by the operation feasibility determination unit and response information output from the inference runtime for the original model is equal to or less than a threshold, and selects, from among the models determined to be operable, a model for which the error is equal to or less than the threshold as a candidate model for machine learning.

17. The method of claim 16.

Images