Deep learning model deployment method and apparatus

Abstract

The application provides a deep learning model deployment method and device, the method comprising: obtaining a deep learning original model file, a first relationship table and a second relationship table; the first relationship table containing each original operator in the deep learning original model file and a first intermediate representation operator set version of each original operator; the second relationship table including the first intermediate representation operator set version, a corresponding relationship between a first inference engine version corresponding to the first intermediate representation operator set version and a first intermediate representation version; determining a version parameter required for deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table; determining a deployable intermediate representation model of the deep learning original model based on the version parameter; and deploying the deployable intermediate representation model on a target device. Based on this, the problem that a deep learning model deployment process needs to rely on a large number of manual operations, is not convenient and is not automated is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a deep learning model deployment method and device. BACKGROUND

[0002] In recent years, with the continuous improvement of hardware performance such as graphics processing units (GPUs), central processing units (CPUs), neural network processing units (NPUs), and the like, artificial intelligence technologies represented by deep learning have made rapid progress and are widely used in various industries. In order to facilitate developers to quickly and efficiently implement intelligent algorithms based on deep learning, deep learning programming frameworks such as Caffe, TensorFlow, PyTorch, MXNet, and PaddlePaddle have emerged. The emergence of numerous deep learning development frameworks has accelerated model development, but the environments necessary for running the models generated by development and training in different frameworks are not the same, which makes it necessary to install a large number of different dependent software and perform corresponding environment configuration when models developed based on different frameworks need to be deployed together, thereby causing the work of preparing the model running environment to be complicated, time-consuming and laborious. On the other hand, with the continuous expansion of artificial intelligence application scenarios, in addition to traditional servers, devices that need to run deep learning models have also become increasingly diverse, such as mobile phones, development boards, and the like. Due to the large size of the framework, the complex dependent environment, and other reasons, models developed based on deep learning development frameworks are often not suitable for deployment in mobile phones, development boards, and other application environments. In addition, deep learning models have complex structures and large parameters, and real-time running requires high computing power, so when deploying deep learning models in production environments, optimization needs to be performed for the software and hardware environment to maximize the running efficiency of the model. However, the diversity of hardware in the production environment makes it necessary to perform different optimizations for the same deep learning model when it is deployed in different environments, which also brings great challenges to model deployment work.

[0003] Currently, in view of many problems existing in the deployment process of a deep learning model, the academic and industrial circles have constructed a model deployment pipeline through years of exploration. By converting an original model developed and trained by a deep learning development framework into an intermediate representation only describing a network structure, and then converting the intermediate representation into a specific file format by an inference engine and running the model on a corresponding hardware platform, the deep learning model can be quickly and efficiently deployed on an environment. However, there are still some problems in the process, for example, when the original model developed and trained by the deep learning development framework is converted into the intermediate representation, each operator in the original model needs to be translated into an operator defined by the intermediate representation, and in the translation process, there may be an operator incompatibility problem. For another example, the first intermediate representation version used when the original model is converted into the intermediate representation may be incompatible with the inference engine version in the actual deployment environment. In view of these problems, in the model deployment process, the problems usually need to be manually determined and excluded one by one, so that the deep learning model deployment process still needs to rely on a large amount of manual operation, which is not convenient and automatic. SUMMARY

[0004] The present application provides a deep learning model deployment method and device, which provides a deep learning model deployment method to solve the problem that the deep learning model deployment process still needs to rely on a large amount of manual operation, which is not convenient and automatic.

[0005] The present application provides a deep learning model deployment method, which comprises: obtaining a deep learning original model file, a first relationship table and a second relationship table; wherein the first relationship table comprises a corresponding relationship between each original operator in the deep learning original model file, a first intermediate representation operator set version corresponding to each original operator, and a first development framework version; the second relationship table comprises a corresponding relationship between the first intermediate representation operator set version, a first inference engine version corresponding to the first intermediate representation operator set version, and a first intermediate representation version; the first intermediate representation operator set version is an operator set version number supported by the corresponding intermediate representation version; based on the deep learning original model file, the first relationship table and the second relationship table, determining the version parameters required for deep learning original model conversion; based on the version parameters, determining a deployable intermediate representation model of the deep learning original model in the deep learning original model file; deploying the deployable intermediate representation model on a target device.

[0006] In any of the above embodiments, determining the version parameters required for the deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table comprises: determining the first intermediate representation operator set version and the first development framework version corresponding to each original operator based on the deep learning original model file and the first relationship table; determining the first inference engine version and the first intermediate representation version corresponding to each original operator based on the first intermediate representation operator set version corresponding to each original operator and the second relationship table; and determining the version parameters required for the deep learning original model conversion based on the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator.

[0007] In any of the above embodiments, determining the first intermediate representation operator set version and the first development framework version corresponding to each original operator based on the deep learning original model file and the first relationship table comprises: determining each original operator in the deep learning original model based on the deep learning original model file; and obtaining the first intermediate representation operator set version and the first development framework version corresponding to each original operator from the first relationship table.

[0008] In any of the above embodiments, determining the version parameters required for the deep learning original model conversion based on the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator comprises: determining the second intermediate representation operator set version required for the deep learning original model conversion as the first intermediate representation operator set version with the highest version requirement among the first intermediate representation operator set versions corresponding to each original operator; the second intermediate representation operator set version being the operator set version supported by the intermediate representation version required for the deep learning original model conversion; sequentially determining the second development framework version, the second inference engine version and the second intermediate representation version corresponding to the second intermediate representation operator set version from the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator; and determining the second intermediate representation operator set version, the second development framework version, the second inference engine version and the second intermediate representation version as the version parameters required for the deep learning original model conversion, and showing the version parameters to the user.

[0009] In any of the above embodiments, determining, based on the version parameter, a deployable intermediate representation model of the deep learning original model in the deep learning original model file includes: in response to a user operation, installing a target deep learning development framework version; wherein the target deep learning development framework version is the second development framework version or a deep learning development framework version higher than the second development framework version; calling an intermediate representation conversion interface corresponding to the second intermediate representation version provided by the target deep learning development framework version, taking a parameter format corresponding to the second intermediate representation operator set version as a parameter input format of the intermediate representation conversion interface, to obtain an initial intermediate representation model of the deep learning original model; based on the inference engine corresponding to the second inference engine version, performing format verification on the initial intermediate representation model, and in a case where the format verification passes, saving the initial intermediate representation model as a deployable intermediate representation model; or in a case where the format verification fails, outputting a format verification error.

[0010] In any of the above embodiments, before the format verification of the initial intermediate representation model based on the inference engine corresponding to the second inference engine version, the method further includes: in a case where the inference engine version of a target environment is lower than or higher than the second inference engine version, issuing a target prompt to a user, the target prompt being used to remind the user that the inference engine version is too low and that the inference engine of the second inference engine version needs to be installed.

[0011] In any of the above embodiments, before the obtaining of the deep learning original model file, the first relationship table and the second relationship table, the method further includes: based on the relationship between each operator in each deep learning development framework version and a third intermediate representation operator set version, determining and saving the first relationship table; wherein the third intermediate representation operator set version is an intermediate representation operator set version supported by each deep learning development framework in a file under a directory corresponding to each deep learning development framework version; based on release information of inference engine versions, intermediate representation versions and intermediate representation operator set versions, determining and saving the second relationship table.

[0012] The application further provides a deep learning model deployment device, comprising: an acquisition module, configured to acquire a deep learning original model file, a first relationship table and a second relationship table; wherein the first relationship table is used to represent the correspondence among an original operator, a first intermediate representation operator set version and a first development framework version; the second relationship table comprises the first intermediate representation operator set version, and the correspondence among the first intermediate representation operator set version, a first inference engine version corresponding to the first intermediate representation operator set version and a first intermediate representation version; the first intermediate representation operator set version is an operator set version number supported by a corresponding intermediate representation version; a first determination module, configured to determine a version parameter required for deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table; a second determination module, configured to determine a deployable intermediate representation model of a deep learning original model in the deep learning original model file based on the version parameter; and a deployment module, configured to deploy the deployable intermediate representation model on a target device.

[0013] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the deep learning model deployment method according to any one of the above when executing the program.

[0014] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the deep learning model deployment method according to any one of the above.

[0015] The deep learning model deployment method and device provided by the application can automatically obtain the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each operator of the deep learning original model according to the relationships in the first relationship table and the second relationship table during the deployment process, and finally determine the intermediate representation operator set version, the inference engine version and the intermediate representation version used when performing intermediate representation on the deep learning original model based on the information. Therefore, the dependence on manual operation is reduced, the determination of the version parameter required for deep learning original model conversion is automatically implemented, and the deployment process is more convenient and automatic. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to make the technical solutions in the present application or prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0017] Figure 1 is one of the flowcharts of the deep learning model deployment method provided by the present application;

[0018] Figure 2 is another flowchart of the deep learning model deployment method provided by the present application;

[0019] Figure 3 is a third flowchart of the deep learning model deployment method provided by the present application;

[0020] Figure 4 is a structural schematic diagram of the deep learning model deployment device provided by the present application;

[0021] Figure 5 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0022] In order to make the technical solutions in the present application or prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0023] In order to facilitate understanding, the technical terms involved in the present application are first explained.

[0024] Deep learning development framework: in the initial stage of deep learning, each deep learning researcher needs to write a large amount of repetitive code. In order to improve work efficiency, these researchers write these codes into a framework for all researchers to use. The most popular frameworks currently include Tensorflow, PyTorch, MXNet and PaddlePaddle.

[0025] In order to facilitate understanding, the inventive concept of the present application is first described.

[0026] In the face of many problems existing in the deployment process of deep learning models, the academic and industrial communities have constructed a model deployment pipeline after years of exploration. In order to enable deep learning models to be quickly and efficiently deployed on a certain environment, developers can first use any deep learning development framework to define the network structure and determine the parameters in the network through training to produce an original model file. Currently popular deep learning development frameworks include TensorFlow, PyTorch, MXNet, PaddlePaddle, etc. Then, the original model is converted into an intermediate representation that only describes the network structure, and some optimizations are performed on the network structure during the conversion to the intermediate representation. Common intermediate representations include ONNX, TorchScript, Caffe, etc. Finally, the inference engine converts the intermediate representation into a specific file format and runs the model on the corresponding hardware platform. The mainstream inference engines include ONNX Runtime, TensorRT, OpenVINO, NCNN, etc. The inference engine is written using a hardware-oriented high-performance programming framework, which can optimize the execution of operators in the deep learning network model and improve the running efficiency of the entire model.

[0027] However, although the deep learning model deployment pipeline provides process specifications for model deployment, and the mainstream deep learning development frameworks generally provide functions to convert to intermediate representations, simply calling the model conversion function to convert the original model to a certain intermediate representation and then deploying it in the production environment often cannot guarantee that the model can run correctly. There are still various problems to be solved in the process from the original model to the correctly deployed model. For example, when converting the original model developed and trained by the deep learning development framework to an intermediate representation, each operator in the original model needs to be translated into an operator defined by the intermediate representation, and there may be operator incompatibility problems during the translation process. For another example, the first intermediate representation version used when converting the original model to an intermediate representation may be incompatible with the inference engine version in the actual deployment environment. And to solve these problems, in general, manual operations are needed to determine and exclude these problems one by one, so that the deep learning model deployment process still needs to rely on a large number of manual operations, which is not convenient and automated. Moreover, in the case where the original model can be successfully converted to an intermediate representation, the format of the intermediate representation file may not be correct, and corresponding verification is also performed to enable the original model to be converted to an intermediate representation file with the correct format.

[0028] Therefore, in order to more quickly and conveniently and automatically convert the original model into a correct intermediate representation and successfully deploy in a specific environment, the present application provides a deep learning model deployment method, according to the original model, the intermediate representation and the basic information of the inference engine in the target environment, automatically determines how to correctly convert the original model into an intermediate representation capable of running on the target inference engine, calls the related interface provided by the deep learning development framework and sets the optimal parameters to convert the original model into the intermediate representation; and then automatically verifies the correctness of the intermediate representation format.

[0029] The deep learning model deployment method and device of the present application will be described below in conjunction with the accompanying drawings.

[0030] Figure 1 Fig. 1 is a flowchart of the deep learning model deployment method provided by the present application. The deep learning model deployment method can be executed by a deep learning model deployment device in a target intelligent agent, and the deep learning model deployment device can be a computer device.

[0031] It can be understood that, in one of the embodiments, as shown in Fig. 1, before the step 110 of obtaining the deep learning original model file, the first relationship table and the second relationship table, the deep learning model deployment method can include steps 101 and 102: Figure 1

[0032] The step 101 determines and saves the first relationship table based on the relationship between each operator in each deep learning development framework version and the third intermediate representation operator set version.

[0033] The third intermediate representation operator set version is the intermediate representation operator set version supported by each deep learning development framework in the file under the directory corresponding to each deep learning development framework version.

[0034] Specifically, the intermediate representation operator set version supported by the deep learning development framework can be determined from the file under the directory corresponding to each deep learning development framework version. For example, the relevant files under the torch.onnx directory in the PyTroch framework can be used to obtain the version of the ONNX operator set that PyTorch can support, wherein the relevant files can be symbolic_opset{n}.py, which represents the new content added by PyTorch when supporting the nth version of the ONNX operator set, for example, the operator grid_sample in the PyTorch 1.12 version supports the ONNX operator set version 16; and then the corresponding original operator and ONNX operator set version under the corresponding version of the PyTorch development framework can be determined, so that the first relationship table can be determined and saved. Table 1 shows an example of the first relationship table.

[0035] Table 1​

[0036]

[0037] In Table 1, the PyTorch development framework and the ONNX intermediate representation are taken as examples to exemplarily give the PyTorch development framework version and the ONNX intermediate representation version corresponding to the original operator grid_sample and einsum.

[0038] It can be understood that Table 1 only exemplarily gives the development framework version and the intermediate representation version corresponding to two original operators, and in actual cases, the first relationship table can include more development framework versions and intermediate representation versions corresponding to other original operators.

[0039] In step 102, based on the publishing information of the inference engine version, the intermediate representation version and the intermediate representation operator set version, the second relationship table is determined and saved.

[0040] That is, the second relationship table can be pre-constructed and saved according to the publishing information of the inference engine version, the intermediate representation version and the intermediate representation operator set version. Table 2 takes the ONNX Runtime version, the ONNX version and the ONNX operator set version as examples to give an example of the second relationship table.

[0041] Table 2

[0042]

[0043] In Table 2, the ONNX version and the ONNX operator set version corresponding to the ONNX Runtime version are exemplarily given.

[0044] It can be understood that Table 2 only exemplarily gives two corresponding relationships between the ONNX Runtime version, the ONNX version and the ONNX operator set version, and in actual cases, the second relationship table can include more relationships between the ONNX Runtime version, the ONNX version and the ONNX operator set version.

[0045] In step 110, the deep learning original model file, the first relationship table and the second relationship table are obtained.

[0046] In the first relationship table, the corresponding relationship between each original operator in the deep learning original model file, the first intermediate representation operator set version corresponding to each original operator and the first development framework version is included. In the second relationship table, the first intermediate representation operator set version, the corresponding relationship between the first inference engine version and the first intermediate representation version corresponding to the first intermediate representation operator set version are included.

[0047] It can be understood that the deep learning original model is a model file developed and trained using a deep learning development framework. The deep learning original model mainly consists of one original operator after another, and one original operator is equivalent to a function that completes a specific calculation. Taking the model file test.pth developed and trained using the PyTorch deep learning development framework as an example, the original operators may include, for example, Conv2d, MaxPool, ReLU, Transpose, Reshape, MatMul, Pad, and Grid_Sample. The first development framework version is the deep learning development framework version for training the deep learning original model.

[0048] The first intermediate representation version may be a version corresponding to the intermediate representation, such as a version corresponding to any one of ONNX, TorchScript, and Caffe. The first intermediate representation operator set version is a version number of an operator set supported by the corresponding intermediate representation version, such as an ONNX operator set version “16” corresponding to an ONNX version “1.11”. The first inference engine version is a runtime version corresponding to the intermediate representation, such as an ONNX Runtime version.

[0049] In step 120, version parameters required for deep learning original model conversion are determined based on the deep learning original model file, the first relationship table, and the second relationship table.

[0050] The version parameters required for deep learning original model conversion include a development framework version, an inference engine version, an intermediate representation version, and an intermediate representation operator set version required for deep learning original model conversion.

[0051] It can be understood that, according to the correspondence among the original operators corresponding to the pre-saved deep learning original model file, the first intermediate representation operator set version, and the first development framework version, the corresponding first intermediate representation operator set version and the first development framework version can be determined according to the deep learning original model file and the corresponding first relationship table in the actual query process, thereby realizing the automation of the first intermediate representation operator set version and the first development framework version determination process. According to the release information of the inference engine version, the intermediate representation version, and the intermediate representation operator set version, the second relationship table is pre-constructed and saved, so that the corresponding inference engine version and intermediate representation version can be determined by the computer according to the intermediate representation operator set version and the second relationship table in the actual query process, thereby realizing the automation of the inference engine version and the intermediate representation version determination process.

[0052] In step 130, a deployable intermediate representation model of the deep learning original model in the deep learning original model file is determined based on the version parameters.

[0053] It can be understood that after determining the version parameters of the development framework version, the inference engine version, the intermediate representation version, and the intermediate representation operator set version required for deep learning original model conversion, the development framework environment required for deep learning original model conversion can be further determined according to the version parameters, and the deep learning original model file is converted into a deployable intermediate representation model of the deep learning original model using the intermediate representation version and the intermediate representation operator set version, thereby reducing the conversion process error of the deployable intermediate representation model of the deep learning original model caused by the development framework version, the intermediate representation version, or the intermediate representation operator set version error, and thereby improving the speed of deep learning model deployment to some extent.

[0054] Step 140, deploying the deployable intermediate representation model on a target device.

[0055] Specifically, the deployable intermediate representation model can be deployed on a target device using the inference engine corresponding to the inference engine version in the version parameters.

[0056] The deep learning model deployment method provided by the application can automatically obtain the first intermediate representation operator set version and the first development framework version, the first inference engine version, and the first intermediate representation version corresponding to each operator of the deep learning original model during the deployment process based on the first relationship table and the second relationship table, that is, the correspondence between the original operators in the deep learning original model, the first intermediate representation operator set version, and the first development framework version, and the correspondence between the first inference engine version, the first intermediate representation version, and the first intermediate representation operator set version, and finally determine the intermediate representation operator set version, the inference engine version, and the intermediate representation version used when the deep learning original model is converted based on this information. Therefore, the dependence on manual work is reduced, the determination of the version parameters required for deep learning original model conversion is automatically realized, and the deployment process is more convenient and automated.

[0057] In one embodiment, as shown in Figure 2 determining the version parameters required for deep learning original model conversion based on the deep learning original model file, the first relationship table, and the second relationship table includes the following steps:

[0058] Step 210, determining the first intermediate representation operator set version and the first development framework version corresponding to each original operator based on the deep learning original model file and the first relationship table.

[0059] Specifically, this step can include step 2101 and step 2102.

[0060] Step 2101, based on the deep learning original model file, determining each original operator in the deep learning original model.

[0061] Specifically, the original operator in the deep learning original model file can be recognized by a character recognition algorithm.

[0062] It can be understood that the name of the original operator in the deep learning original model file is fixed, so the original operator contained in the deep learning original model file can be recognized by a character recognition algorithm. The character recognition algorithm can refer to the prior art.

[0063] Step 2102, obtaining the first intermediate representation operator set version and the first development framework version corresponding to each original operator from the first relationship table.

[0064] Exemplarily, if the original operators in the deep learning original model are grid_sample and einsum, it can be obtained from the first relationship table 1 that the first intermediate representation operator set version of grid_sample is the ONNX operator set version “16”, the first development framework version of grid_sample is the PyTorch version “1.12”, the first intermediate representation operator set version of einsum is the ONNX operator set version “12”, and the first development framework version of einsum is the PyTorch version “1.9”.

[0065] Step 220, based on the first intermediate representation operator set version corresponding to each original operator and the second relationship table, determining the first inference engine version and the first intermediate representation version corresponding to each original operator.

[0066] In combination with the example in step 2102, after obtaining the first intermediate representation operator set version of the original operator grid_sample from the first relationship table 1, which is the ONNX operator set version “16”, it can be obtained from the second relationship table 2 that the first inference engine version of the original operator grid_sample is the ONNX Runtime version “1.11”, and the first intermediate representation version of the original operator grid_sample is the ONNX version “1.11”. Similarly, after obtaining the first intermediate representation operator set version of einsum from the first relationship table 1, which is the ONNX operator set version “12”, it can be obtained from the second relationship table that the first inference engine version of the original operator einsum is the ONNX Runtime version “1.3, 1.4, 1.5”, and the first intermediate representation version of the original operator einsum is the ONNX version “1.7”.

[0067] At step 230, based on the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator, a version parameter required for deep learning original model conversion is determined.

[0068] In one embodiment, the step 230 includes step 2301 and step 2302.

[0069] At step 2301, the first intermediate representation operator set version with the highest version requirement in the first intermediate representation operator set versions corresponding to each original operator is determined as the second intermediate representation operator set version required for deep learning original model conversion.

[0070] The second intermediate representation operator set version is the operator set version supported by the intermediate representation version required for deep learning original model conversion.

[0071] In combination with the above examples in steps 210 and 220, the first intermediate representation operator set versions of the two original operators grid_sample and einsum are ONNX operator set version “16” and ONNX operator set version “12” respectively, so the ONNX operator set version “16” with the highest version requirement can be determined as the second intermediate representation operator set version required for deep learning original model conversion.

[0072] It can be understood that through this step, the compatibility of the intermediate representation operator set versions corresponding to each original operator of the deep learning original model can be realized.

[0073] At step 2302, the second development framework version, the second inference engine version and the second intermediate representation version corresponding to the second intermediate representation operator set version are determined in turn from the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator.

[0074] Exemplarily, in combination with the above examples, after determining that the second intermediate representation operator set version is ONNX operator set version “16”, the second inference engine version corresponding to the ONNX operator set version “16” is ONNX Runtime version “1.11”, the second intermediate representation version is ONNX version “1.11”, and the second development framework version is PyTorch version “1.12” can be determined in turn from the first development framework version, the first inference engine version and the first intermediate representation version corresponding to the ONNX operator set version “16”.

[0075] It can be understood that after determining the second intermediate representation operator set version, the second development framework version, the second inference engine version and the second intermediate representation version corresponding to the second intermediate representation operator set version are determined as the version parameters required for deep learning original model conversion. Based on this, the finally determined second intermediate representation operator set version and its corresponding second development framework version, second inference engine version and second intermediate representation version can all be compatible with each original operator. In addition to achieving automatic determination of the version parameters required for deep learning original model conversion, the error caused by the incompatibility of any one of the development framework version, the inference engine version, the intermediate representation version or the intermediate representation operator set version corresponding to each original operator is reduced, thereby speeding up the deployment of the deep learning model to a certain extent.

[0076] Step 2303, determining the second intermediate representation operator set version, the second development framework version, the second inference engine version and the second intermediate representation version as the version parameters required for deep learning original model conversion, and showing the version parameters to the user.

[0077] In one embodiment, as shown in Figure 3 Based on the version parameters, determining the deployable intermediate representation model of the deep learning original model in the deep learning original model file includes:

[0078] Step 310, in response to the operation of the user, installing a target deep learning development framework version; wherein the target deep learning development framework version is the second development framework version or a deep learning development framework version higher than the second development framework version.

[0079] The operation of the user can be the calling of the installation interface corresponding to the target deep learning development framework version and the configuration operation of the installation environment.

[0080] In combination with the above example, if it is determined that the second development framework version is PyTorch version “1.12”, the user can choose to install PyTorch 1.12 and above version environment.

[0081] Step 320, calling the intermediate representation conversion interface provided by the target deep learning development framework version corresponding to the second intermediate representation version, taking the parameter format corresponding to the second intermediate representation operator set version as the parameter input format of the intermediate representation conversion interface, to obtain the initial intermediate representation model of the deep learning original model.

[0082] In combination with the above examples, if it is determined that the second intermediate representation operator set version is ONNX operator set version "16" and the second intermediate representation version is ONNX version "1.11", a conversion interface of the ONNX version "1.11" can be selected, and the version parameter format of the ONNX operator set version "16" is used as the parameter input format of the conversion interface of the ONNX version "1.11".

[0083] In step 330, the initial intermediate representation model is format-verified based on the inference engine corresponding to the second inference engine version. If the format verification is passed, the initial intermediate representation model is saved as a deployable intermediate representation model. If the format verification is not passed, a format verification error is output.

[0084] In combination with the above examples, if it is determined that the second inference engine version is ONNX Runtime version "1.11", the initial intermediate representation model can be format-verified based on the inference engine corresponding to the ONNX Runtime version "1.11". For example, the onnx.cheker.check_model function provided by the PyTorch framework can be used to check whether the model format is correct. If there is an error, the function will directly report an error.

[0085] In one of the embodiments, before the initial intermediate representation model is format-verified based on the inference engine corresponding to the second inference engine version, the method further includes:

[0086] If it is determined that the inference engine version of the target environment is lower than or higher than the second inference engine version, a target prompt is issued to the user, which is used to remind the user that the inference engine version is too low and the inference engine of the second inference engine version needs to be installed.

[0087] In one of the embodiments, after the initial intermediate representation model is saved as a deployable intermediate representation model, the method further includes:

[0088] Based on the requirements of the inference engine of the target environment, the inference engine corresponding to the second inference engine version is imported to build a test verification environment. Test data is automatically generated according to the input parameters required by the deployable intermediate representation model. Based on the test data, the model result of the deployable intermediate representation model is inferred in the test verification environment to obtain a target inference result. The target inference result is compared with the standard inference result of the deep learning original model. If the comparison result is less than a preset threshold, the inference result verification is passed. The converted model is inferred in the automatically constructed target inference engine environment to verify the inference accuracy, so as to ensure that the converted model can correctly run in the deployment environment.

[0089] It can be understood that the present application only exemplarily gives the process of adding an automatic analysis module before the deep learning model conversion, analyzing the possible problems in the conversion and finding the best conversion parameters according to the model file, the development framework, the intermediate representation, the target inference engine and other information. In actual situations, other information can also be used to prepare for subsequent deep learning model automation, such as comparing and analyzing the inference results of the original inference model and the intermediate representation inference results, so as to judge the accuracy of the converted model and ensure that the converted model can correctly run in the deployment environment.

[0090] The deep learning model deployment device provided by the present application is described below, and the deep learning model deployment device described below can be correspondingly referred to the deep learning model deployment method described above. The training device of the deep learning model deployment described below can be correspondingly referred to the training method of the deep learning model deployment described above.

[0091] Figure 4 is a schematic diagram of the deep learning model deployment device provided by the present application, as Figure 4 indicated, the deep learning model deployment device provided by the present application comprises:

[0092] The acquisition module 410 is configured to acquire a deep learning original model file, a first relationship table and a second relationship table. The first relationship table is used to indicate the correspondence between the original operator, the first intermediate representation operator set version and the first development framework version. The second relationship table includes the first intermediate representation operator set version, and the correspondence between the first inference engine version corresponding to the first intermediate representation operator set version and the first intermediate representation version. The first intermediate representation operator set version is the operator set version number supported by the corresponding intermediate representation version.

[0093] The first determination module 420 is configured to determine the version parameter required for the deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table.

[0094] The second determination module 430 is configured to determine the deployable intermediate representation model of the deep learning original model in the deep learning original model file based on the version parameter.

[0095] The deployment module 440 is configured to deploy the deployable intermediate representation model on a target device.

[0096] The device for deploying a deep learning model provided by the application can automatically obtain the first intermediate representation operator set version and the first development framework version, the first inference engine version, and the first intermediate representation version corresponding to each operator of the deep learning original model according to the relationships during the deployment process by pre-storing the first relationship table and the second relationship table, that is, the corresponding relationships among the original operators in the deep learning original model, the first intermediate representation operator set version, and the first development framework version, and the corresponding relationships among the first inference engine version, the first intermediate representation version, and the first intermediate representation operator set version, and finally determining the intermediate representation operator set version, the inference engine version, and the intermediate representation version used when the deep learning original model is subjected to intermediate representation based on the information. Therefore, the dependence on manual work is reduced, the determination of the version parameters required for the conversion of the deep learning original model is automatically implemented, and the deployment process is more convenient and automated.

[0097] Figure 5 An example of a schematic diagram of a physical structure of an electronic device is shown in Figure 5 The electronic device can include a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 can communicate with each other through the communications bus 540. The processor 510 can invoke the logical instructions in the memory 530 to execute a deep learning model deployment method. The deep learning model deployment method includes obtaining a deep learning original model file, a first relationship table, and a second relationship table. The first relationship table includes the corresponding relationships among each original operator in the deep learning original model file, the first intermediate representation operator set version corresponding to each original operator, and the first development framework version. The second relationship table includes the corresponding relationships among the first intermediate representation operator set version, the first inference engine version corresponding to the first intermediate representation operator set version, and the first intermediate representation version. The first intermediate representation operator set version is the operator set version number supported by the corresponding intermediate representation version. Based on the deep learning original model file, the first relationship table, and the second relationship table, the version parameters required for the conversion of the deep learning original model are determined. Based on the version parameters, the deployable intermediate representation model of the deep learning original model in the deep learning original model file is determined. The deployable intermediate representation model is deployed on a target device.

[0098] In addition, the logic instructions in the memory 530 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0099] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the deep learning model deployment method provided by the present application. Wherein, the deep learning model deployment method comprises: obtaining a deep learning original model file, a first relationship table and a second relationship table; wherein, the first relationship table comprises the corresponding relationship between each original operator in the deep learning original model file, the first intermediate representation operator set version corresponding to each original operator and the first development framework version; the second relationship table comprises the first intermediate representation operator set version, and the corresponding relationship between the first inference engine version corresponding to the first intermediate representation operator set version and the first intermediate representation version; the first intermediate representation operator set version is the operator set version number supported by the corresponding intermediate representation version; determining the version parameter required for deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table; determining the deployable intermediate representation model of the deep learning original model in the deep learning original model file based on the version parameter; deploying the deployable intermediate representation model on a target device.

[0100] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the deep learning model deployment method provided by the present application. The deep learning model deployment method comprises: obtaining a deep learning original model file, a first relationship table and a second relationship table; the first relationship table comprises a corresponding relationship between each original operator in the deep learning original model file, a first intermediate representation operator set version corresponding to each original operator, and a first development framework version; the second relationship table comprises a corresponding relationship between the first intermediate representation operator set version, a first inference engine version corresponding to the first intermediate representation operator set version, and a first intermediate representation version; the first intermediate representation operator set version is an operator set version number supported by the corresponding intermediate representation version; determining a version parameter required for deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table; determining a deployable intermediate representation model of a deep learning original model in the deep learning original model file based on the version parameter; and deploying the deployable intermediate representation model on a target device.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0102] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0103] It can be understood that the above embodiments are only used to illustrate the technical solutions of the present application, but not limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A deep learning model deployment method, characterized in that, The method comprises: obtaining a deep learning original model file, a first relationship table and a second relationship table; wherein the first relationship table comprises a corresponding relationship between each original operator in the deep learning original model file, a first intermediate representation operator set version corresponding to each original operator and a first development framework version; the second relationship table comprises a corresponding relationship between the first intermediate representation operator set version, a first inference engine version corresponding to the first intermediate representation operator set version and a first intermediate representation version; the first intermediate representation operator set version is an operator set version number supported by a corresponding intermediate representation version; determining a version parameter required for deep learning original model conversion based on the deep learning original model file, the first relationship table and the second relationship table; determining a deployable intermediate representation model of a deep learning original model in the deep learning original model file based on the version parameter; deploying the deployable intermediate representation model on a target device; The method comprises: determining the first intermediate representation operator set version and the first development framework version corresponding to each original operator based on the deep learning original model file and the first relationship table; determining the first inference engine version and the first intermediate representation version corresponding to each original operator based on the first intermediate representation operator set version corresponding to each original operator and the second relationship table; determining the version parameter required for deep learning original model conversion based on the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator. 2.The deep learning model deployment method of claim 1, wherein, The method comprises: determining each original operator in the deep learning original model based on the deep learning original model file; obtaining the first intermediate representation operator set version and the first development framework version corresponding to each original operator from the first relationship table. 3.The deep learning model deployment method of claim 2, wherein, The method comprises: determining the version parameter required for deep learning original model conversion based on the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator. determining the first intermediate representation operator set version with the highest version requirement in the first intermediate representation operator set version corresponding to each original operator as a second intermediate representation operator set version required for deep learning original model conversion; the second intermediate representation operator set version is an operator set version supported by an intermediate representation version required for deep learning original model conversion; determining the second development framework version, the second inference engine version and the second intermediate representation version corresponding to the second intermediate representation operator set version from the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator in turn. The second intermediate representation operator set version, the second development framework version, the second inference engine version, and the second intermediate representation version are determined as version parameters required for deep learning original model conversion, and the version parameters are displayed to the user. 4.The deep learning model deployment method of claim 3, wherein, The determining, based on the version parameters, of the deployable intermediate representation model of the deep learning original model in the deep learning original model file comprises: In response to an operation of the user, a target deep learning development framework version is installed; the target deep learning development framework version is the second development framework version or a deep learning development framework version higher than the second development framework version; An intermediate representation conversion interface corresponding to the second intermediate representation version provided by the target deep learning development framework version is called, and a parameter format corresponding to the second intermediate representation operator set version is used as a parameter input format of the intermediate representation conversion interface, to obtain an initial intermediate representation model of the deep learning original model; Based on an inference engine corresponding to the second inference engine version, the initial intermediate representation model is subjected to format verification, and in a case where the format verification is passed, the initial intermediate representation model is saved as a deployable intermediate representation model; or in a case where the format verification is not passed, a format verification error is output. 5.The deep learning model deployment method of claim 4, wherein, Before the format verification of the initial intermediate representation model based on the inference engine corresponding to the second inference engine version, the method further comprises: In a case where the inference engine version of the target environment is lower than or higher than the second inference engine version, a target prompt is issued to the user, the target prompt being used to remind the user that the inference engine version is too low and that the inference engine of the second inference engine version needs to be installed. 6.The method of claim 1, wherein, Before the obtaining of the deep learning original model file, the first relationship table, and the second relationship table, the method further comprises: Based on the relationship between each operator in each deep learning development framework version and a third intermediate representation operator set version, the first relationship table is determined and saved; the third intermediate representation operator set version is a version of an intermediate representation operator set supported by each deep learning development framework in a file in a directory corresponding to each deep learning development framework version; Based on release information of inference engine versions, intermediate representation versions, and intermediate representation operator set versions, the second relationship table is determined and saved. 7.A deep learning model deployment apparatus characterized by comprising: The apparatus comprises: An obtaining module is configured to obtain a deep learning original model file, a first relationship table, and a second relationship table; the first relationship table is used to indicate a corresponding relationship among an original operator, a first intermediate representation operator set version, and a first development framework version; the second relationship table includes a corresponding relationship between the first intermediate representation operator set version and a first inference engine version corresponding to the first intermediate representation operator set version and a first intermediate representation version corresponding to the first intermediate representation operator set version; the first intermediate representation operator set version is a version number of an operator set supported by a corresponding intermediate representation version; A first determining module is configured to determine, based on the deep learning original model file, the first relationship table, and the second relationship table, version parameters required for deep learning original model conversion. The second determining module is configured to determine, based on the version parameter, a deployable intermediate representation model of the deep learning original model in the deep learning original model file; The deploying module is configured to deploy the deployable intermediate representation model on a target device; The first determining module is further configured to: determine, based on the deep learning original model file and the first relationship table, a first intermediate representation operator set version and a first development framework version corresponding to each original operator; determine, based on the first intermediate representation operator set version corresponding to each original operator and the second relationship table, a first inference engine version and a first intermediate representation version corresponding to each original operator; determine, based on the first intermediate representation operator set version, the first development framework version, the first inference engine version and the first intermediate representation version corresponding to each original operator, a version parameter required for deep learning original model conversion.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the deep learning model deployment method according to any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the steps of the deep learning model deployment method according to any one of claims 1 to 6 when executed by the processor.

Images