Management method, system, device and electronic equipment for model conversion

Abstract

The embodiments of the present application provide a management method, system, device and electronic device for model conversion. This embodiment reflects both the second deep learning model that matches the target AI device and the model description information used to describe the second deep learning model in the target deep learning model. This can intuitively mark the key information required for deep learning reasoning using the target deep learning model, without the need to additionally record fragmented information such as AI device information, deep learning reasoning framework, model training framework, etc. corresponding to the target deep learning model, thereby achieving unified management of the model conversion process of multiple AI device platforms (platforms that mix AI devices with different application scenarios); further, when the model description information used to describe the second deep learning model is reflected in the target deep learning model, the target deep learning model can be subsequently upgraded based on the model description information.

Description

Technical Field

[0001] The present application relates to computer technology, and in particular to a management method, system, device and electronic equipment for model conversion. Background Art

[0002] AI devices can be used for deep learning training and deep learning reasoning. Deep learning training refers to the process of feeding a set of training data sets (labeled data) into a neural network, adjusting the connection weights of each network layer in the neural network based on the difference between the output information of the neural network and the preset expected output information, and ultimately training an ideal neural network. Deep learning reasoning refers to the process of using a trained neural network to perform data reasoning, such as recognizing images, recognizing recorded spoken language, detecting diseases in the blood, or recommending clothing that suits a person's style. As an embodiment, an AI device may be a device that has an AI chip deployed, and the AI ​​chip may be used to implement the above-mentioned deep learning training and deep learning reasoning.

[0003] Compared to deep learning training, deep learning inference requires much higher performance. To fully utilize the high performance of AI devices, AI device manufacturers configure their AI devices with corresponding deep learning inference frameworks.

[0004] Currently, deploying deep learning models on AI devices often results in the following issues: The deep learning model and the AI ​​device are incompatible (for example, the deep learning model and the AI ​​device's corresponding deep learning inference framework do not match, making it impossible to perform learning and inference based on the AI ​​device's corresponding deep learning inference framework). Once the deep learning model and the AI ​​device are incompatible, the deep learning model cannot rely on the AI ​​device's corresponding deep learning inference framework for learning and inference. Summary of the Invention

[0005] The present application provides a management method, system, device and electronic device for model conversion to manage the conversion of deep learning models when the deep learning models do not match the AI ​​devices, so that the converted deep learning models match the AI ​​devices.

[0006] The method provided in this application includes:

[0007] A management method for model conversion, the method comprising:

[0008] Obtaining model conversion information of a first deep learning model to be converted, the model conversion information including at least: the first deep learning model and AI device information of a target AI device; the target AI device is an AI device on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device;

[0009] Converting the first deep learning model to obtain a second deep learning model that matches the AI ​​device information of the target AI device;

[0010] A target deep learning model of the first deep learning model is generated based on the second deep learning model and the model description information used to describe the second deep learning model.

[0011] As an embodiment, converting the first deep learning model includes:

[0012] The model conversion information is forwarded to the target AI device corresponding to the AI ​​device information, so that the target AI device converts the first deep learning model through the configured model conversion tool.

[0013] As an embodiment, generating a target deep learning model of the first deep learning model based on the second deep learning model and the model description information used to describe the second deep learning model includes:

[0014] Encapsulating the model description information according to a specified model management protocol to obtain model encapsulation information;

[0015] Loading the model encapsulation information into a specified location of the second deep learning model;

[0016] The second deep learning model with the model encapsulation information loaded at the specified location is packaged to obtain the target deep learning model.

[0017] As an embodiment, encapsulating the model description information according to a specified model management protocol to obtain model encapsulation information includes:

[0018] Encapsulating the model description information and the model conversion information according to a specified model management protocol to obtain the model encapsulation information; or,

[0019] The model description information and information in the model conversion information except the first depth model are encapsulated according to a specified model management protocol to obtain the model encapsulation information.

[0020] As an embodiment, the software version information includes at least one of the following information:

[0021] The minimum version information of the inference software stack used to perform deep learning reasoning using the second deep learning model, and the type of the inference software stack used to perform deep learning reasoning using the second deep learning model.

[0022] As an embodiment, the model parameters include at least one of the following parameters:

[0023] The model weight data format corresponding to the second deep learning model;

[0024] The model type of the second deep learning model; the model type includes at least: a quantization type for indicating that the second deep learning model is a quantized model;

[0025] When the model type of the second deep learning model is a quantization type, the quantization parameter corresponding to the second deep learning model.

[0026] As an embodiment, the method further comprises:

[0027] The target deep learning model is encrypted according to a set encryption algorithm and the encrypted target deep learning model is stored.

[0028] A management system for model conversion, the system comprising:

[0029] A client, a network model management server, and N AI devices; wherein the N AI devices include at least one cloud-side AI device, and / or at least one terminal-side AI device, and / or at least one network edge-side AI device; N is greater than or equal to 1; the client and the N AI devices are respectively connected to the network model management server via a network;

[0030] The network model management server obtains, from the client, model conversion information of a first deep learning model to be converted, the model conversion information including at least the first deep learning model and AI device information of a target AI device; the target AI device is an AI device among the N AI devices on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device; and

[0031] The first deep learning model is converted to obtain a second deep learning model that matches the AI ​​device information of the target AI device, and a target deep learning model of the first deep learning model is generated based on the second deep learning model and the model description information used to describe the second deep learning model.

[0032] A management device for model conversion, applied to a network model management server, comprising:

[0033] an acquiring unit, configured to acquire model conversion information of a first deep learning model to be converted, the model conversion information including at least: the first deep learning model and AI device information of a target AI device; the target AI device is an AI device on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device;

[0034] A first processing unit, configured to convert the first deep learning model to obtain a second deep learning model that matches the target AI device;

[0035] A second processing unit is configured to generate a target deep learning model of the first deep learning model based on the second deep learning model and model description information used to describe the second deep learning model.

[0036] An electronic device comprising: a processor and a machine-readable storage medium;

[0037] The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0038] The processor is used to execute machine-executable instructions to implement the method steps disclosed above.

[0039] As can be seen from the above technical solutions, in this application, this embodiment implements conversion management of the first deep learning model when the first deep learning model does not match the target AI device by embodying the second deep learning model that matches the target AI device in the target deep learning model, so as to make the target deep learning model match the target AI device;

[0040] Furthermore, this embodiment reflects both the second deep learning model that matches the target AI device and the model description information used to describe the second deep learning model in the target deep learning model. This can intuitively mark the key information required for deep learning reasoning using the target deep learning model (specifically, the model description information described above). There is no need to additionally record fragmented information such as the AI ​​device information, deep learning reasoning framework, and model training framework corresponding to the target deep learning model, thereby achieving unified management of the model conversion process for multiple AI device platforms (platforms that mix AI devices for different application scenarios);

[0041] Furthermore, when the model description information used to describe the second deep learning model is reflected in the target deep learning model, the target deep learning model can be subsequently upgraded based on the model description information. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] Figure 1 A schematic diagram of model conversion provided in an embodiment of the present application;

[0044] Figure 2 A flow chart of the method provided in the embodiment of the present application;

[0045] Figure 3 Flowchart for implementing step 203 provided in the embodiment of the present application;

[0046] Figure 4 A system structure diagram provided for an embodiment of the present application;

[0047] Figure 5 A diagram of the device structure provided in an embodiment of the present application;

[0048] Figure 6 This is a hardware structure diagram of the device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0050] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0051] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0052] As an example, when the deep learning model to be deployed does not match the AI ​​device, the deep learning model is converted so that the converted deep learning model matches the AI ​​device. In one example, the conversion of the deep learning model can be achieved with the help of a configured model conversion tool. The following example describes how the model conversion tool converts the deep learning model, which is not detailed here.

[0053] Take the following three deep learning models as an example: model 101a (the corresponding training framework is Caffe, also known as the Caffe model), model 102a (the corresponding training framework is Tensorflow, also known as the Tensorflow model), and model 103a (the corresponding training framework is Pytorch, also known as the Pytorch model). If these three deep learning models need to be deployed on the following three AI devices: AI device 101b, AI device 102b, and AI device 103b, then:

[0054] When model 101a does not match AI device 101b (for example, the deep learning inference framework corresponding to model 101a and AI device 101b (specifically, AI chip 0 on AI device 101b) does not match), model 101a is converted to obtain a model (denoted as model 101c) that matches the deep learning inference framework corresponding to AI device 101b (specifically, AI chip 0 on AI device 101b). Model 101c is then deployed on AI device 101b (specifically, chip 0 on AI device 101b).

[0055] When model 101a does not match AI device 102b (for example, the deep learning inference framework corresponding to model 101a and AI device 102b (specifically, AI chip_1 on AI device 102b) does not match), model 101a is converted to obtain a model (denoted as model 102c) that matches the deep learning inference framework corresponding to AI device 102b (specifically, AI chip_1 on AI device 102b). Model 102c is then deployed on AI device 102b (specifically, chip_1 on AI device 102b).

[0056] When model 101a does not match AI device 103b (for example, the deep learning inference framework corresponding to model 101a and AI device 103b (specifically, AI chip_2 in AI device 103b) does not match), model 101a is converted to obtain a model (denoted as model 103c) that matches the deep learning inference framework corresponding to AI device 103b (specifically, AI chip_2 in AI device 103b). Model 103c is then deployed on AI device 103b (specifically, chip_2 in AI device 103b).

[0057] Similar to model 101a, model 102a and model 103a are deployed on AI devices 101b, 102b, and 103b using a processing method similar to model 101a, and will not be described in detail here.

[0058] From the above description, we can know that by performing model conversion on the deep learning model so that the converted deep learning model matches the AI ​​device, the matching deep learning model can be configured on each AI device.

[0059] It should be noted that, in one example, the above Figure 1 The AI ​​devices 101b to 103b shown are not limited to AI devices in the same application scenario, and can be AI devices used in different application scenarios such as the cloud, terminal side, and network edge side.

[0060] For a platform that mixes AI devices in different application scenarios (referred to as a multi-AI device platform), as an embodiment, after completing the conversion of the deep learning model, for example, after completing the model conversion of model 101a to obtain a model (denoted as model 101c) that matches the deep learning inference framework corresponding to AI device 101b (specifically, AI chip _0 on AI device 101b), it is necessary to further uniformly manage the converted deep learning model to enable the AI ​​device to finally deploy its matching deep learning model. See below for details. Figure 2 The process description is shown.

[0061] In order to achieve unified management of the converted deep learning model, this application provides Figure 2 The process shown.

[0062] See also Figure 2 , Figure 2 This is a flow chart of a method provided in an embodiment of the present application. This process is applied to a network model management server. In one example, the network model management server here can be a newly added device between the client and the AI ​​device. In another example, the network model management server here can also be integrated into an existing device such as a client, an AI device, etc., which is not specifically limited in this embodiment.

[0063] like Figure 2 As shown, the process may include the following steps:

[0064] Step 201: Obtain model conversion information of a first deep learning model to be converted, where the model conversion information includes at least: the first deep learning model and AI device information of a target AI device; the target AI device is an AI device on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device.

[0065] It should be noted that, here, the first deep learning model is named for ease of description and is not intended to be limiting. In one example, the first deep learning model may be an initially trained deep learning model.

[0066] As an embodiment, when the user determines to deploy the above-mentioned first deep learning model on a certain AI device (recorded as the target AI device) according to actual needs, if it is found that the first deep learning model does not match the target AI device, a first model conversion request will be sent to the network model management server through the client, and the first model conversion request carries the above-mentioned model conversion information. At this point, the network model management server obtains the model conversion information of the first deep learning model to be converted. Optionally, here, the mismatch between the first deep learning model and the target AI device may be: the training framework corresponding to the first deep learning model does not match the training framework corresponding to the target AI device, or the inference software stack version information of the deep learning inference framework running the first deep learning model does not match the inference software stack version information corresponding to the target AI device, etc.

[0067] Based on the description of the first model conversion request above, as an embodiment, in step 201, obtaining model conversion information of the first deep learning model to be converted may include:

[0068] A first model conversion request is received from a client, where the first model conversion request carries the model conversion information; and the model conversion information is obtained from the first model conversion request.

[0069] As described in step 201, optionally, the model conversion information includes at least: a first deep learning model, and AI device information of the target AI device. As an embodiment, the AI ​​device information of the target AI device may include: a device identifier of the target AI device (such as the main model and sub-model of the AI ​​chip on the target AI device), device capability information such as the computing instruction set supported by the target AI device. Optionally, when the target AI device includes multiple AI chip sub-models, the device capability information here may be the capability information (software and hardware information) supported by a specified AI chip on the target AI device. In an example, the specified AI chip here may be the AI ​​chip with the highest version capability level of software and hardware information (i.e., chip performance) on the target AI device. The reason for setting the above-mentioned device capability information to the highest version capability level of software and hardware information is that the highest version capability level of software and hardware information has a high degree of redundancy in computing instruction set and software stack, which can effectively save the software and hardware resources required for model conversion.

[0070] Step 202: Convert the first deep learning model to obtain a second deep learning model that matches the AI ​​device information of the target AI device.

[0071] As an embodiment, in step 202, the model conversion information may further include: a training framework corresponding to the first deep learning model. In one example, the training framework corresponding to the first deep learning model may be Caffe, Tensorflow, Pytorch, etc., which is not specifically limited in this embodiment.

[0072] Optionally, in this embodiment, the training framework corresponding to the first deep learning model can be obtained simultaneously with the above-mentioned first deep learning model, or can be obtained in sequence with the above-mentioned first deep learning model, which is not specifically limited in this embodiment.

[0073] As an embodiment, the conversion of the first deep learning model in this step 202 can be implemented by the network model management server. Specifically, the network model management server can determine the corresponding model conversion mechanism based on the training framework corresponding to the first deep learning model in the above-mentioned model conversion information and the AI ​​device information of the target AI device. Thereafter, the first deep learning model is converted based on the above-mentioned model conversion mechanism using the previously acquired model conversion tool corresponding to the target AI device to obtain a second deep learning model that matches the target AI device. Here, the model conversion tool corresponding to the target AI device is used to perform model conversion on the deep learning model to be deployed on the target AI device when the deep learning model to be deployed on the target AI device does not match the target AI device, so as to obtain a deep learning model that matches the target AI device.

[0074] Optionally, in one embodiment, the network model management server converts the first deep learning model through the previously acquired model conversion tool corresponding to the target AI device and based on the above-mentioned model conversion mechanism, which may include: when the training framework corresponding to the first deep learning model does not match the training framework supported by the target AI device, the training framework corresponding to the first deep learning model is converted through the previously acquired model conversion tool corresponding to the target AI device and based on the above-mentioned model conversion mechanism, so that the training framework corresponding to the converted first deep learning model matches the training framework supported by the target AI device (that is, a second deep learning model that matches the target AI device is obtained).

[0075] Optionally, in another embodiment, the network model management server converting the first deep learning model based on the above-mentioned model conversion mechanism using a previously acquired model conversion tool corresponding to the target AI device may include:

[0076] When the first deep learning model does not match the deep learning reasoning framework supported by the target AI device (for example, the first algorithm of the first deep learning model does not match the deep learning reasoning framework supported by the target AI device, and the first deep learning model cannot be run for deep learning reasoning based on the deep learning reasoning framework supported by the target AI device, etc.), the first deep learning model is converted based on the previously obtained model conversion tool corresponding to the target AI device and the above-mentioned model conversion mechanism, so that the converted deep learning model is run based on the deep learning reasoning framework supported by the target AI device for deep learning reasoning (that is, a second deep learning model that matches the target AI device is obtained).

[0077] As another embodiment, in order to reduce the pressure on the network model management server, the conversion of the first deep learning model in this step 202 can also be implemented by the target AI device, specifically: forwarding the above-mentioned model conversion information to the above-mentioned target AI device, so that the target AI device determines the corresponding model conversion mechanism based on the AI ​​device information and training framework in the model conversion information, and converts the first deep learning model through the configured above-mentioned model conversion tool and based on the above-mentioned model conversion mechanism. Here, the target AI device converts the first deep learning model through the configured above-mentioned model conversion tool and based on the above-mentioned model conversion mechanism. Please refer to the description of the conversion of the first deep learning model by the above-mentioned network model management server, which will not be repeated here.

[0078] Finally, a second deep learning model that matches the AI ​​device information of the target AI device is obtained through step 202. It should be noted that in order to uniformly manage the model conversion process of multiple AI device platforms (platforms that mix AI devices with different application scenarios), the second deep learning model at this time is not the final target deep learning model. This embodiment requires further processing of the second deep learning model, as shown in step 203.

[0079] Step 203: Generate a target deep learning model of the first deep learning model based on the second deep learning model and the model description information used to describe the second deep learning model.

[0080] Optionally, in this step 203, the above-mentioned model description information at least includes: software version information used to run the second deep learning model for deep learning reasoning and / or model parameters of the second deep learning model.

[0081] Optionally, as an embodiment, the software version information includes at least one of the following: a minimum version of the inference software stack used for deep learning inference using the second deep learning model, and a type of inference software stack used for deep learning inference using the second deep learning model. In one example, to facilitate tracking and tracing the target deep learning model, the software version information may further include software version information of the aforementioned model conversion tool.

[0082] Optionally, as an embodiment, the above-mentioned model parameters may be some key parameters used to describe the second deep learning model, which will be described with examples below and will not be elaborated here.

[0083] As an embodiment, in the above step 203, there are many implementation methods for generating a target deep learning model of the first deep learning model based on the second deep learning model and the model description information used to describe the second deep learning model. However, no matter which implementation method is used, the target deep learning model finally obtained reflects the second deep learning model and the model description information used to describe the second deep learning model. Figure 3 The process shown is an example to describe one implementation method of the above step 203, which will not be described here in detail.

[0084] This embodiment reflects the model description information used to describe the second deep learning model in the target deep learning model. Its purpose is to intuitively mark the key information required for deep learning reasoning using the target deep learning model (specifically, the model description information described above). There is no need to additionally record fragmented information such as the AI ​​device information, deep learning reasoning framework, and model training framework corresponding to the target deep learning model, thereby achieving unified management of the model conversion process of multiple AI device platforms (platforms that mix AI devices for different application scenarios).

[0085] Furthermore, when the model description information used to describe the second deep learning model is reflected in the target deep learning model, the target deep learning model can be subsequently upgraded based on the model description information.

[0086] So far, completed Figure 2 The process shown.

[0087] pass Figure 2 As shown in the process, this embodiment ultimately obtains a match between the target deep learning model and the target AI device, which implements conversion management of the first deep learning model when the first deep learning model does not match the target AI device, so that the target deep learning model matches the target AI device.

[0088] Furthermore, in this embodiment, the target deep learning model reflects both the second deep learning model that matches the target AI device and the model description information used to describe the second deep learning model. This can intuitively mark the key information required for deep learning reasoning using the target deep learning model (specifically, the model description information described above), eliminating the need to additionally record fragmented information such as the AI ​​device information, deep learning reasoning framework, and model training framework corresponding to the target deep learning model. This enables unified management of the model conversion process for multiple AI device platforms (platforms that mix AI devices for different application scenarios).

[0089] Furthermore, when the model description information used to describe the second deep learning model is reflected in the target deep learning model, the target deep learning model can be subsequently upgraded based on the model description information.

[0090] It should be noted that, in the above step 201, the model conversion information may further include: an optimization processing identifier for optimizing the first deep learning model. Optionally, the optimization processing identifier here may be: an identifier for indicating that the model is optimized, such as quantization and / or compression. When the above-mentioned optimization processing identifier is obtained, the optimization processing corresponding to the optimization processing identifier can be performed in the process of converting the first deep learning model in the above step 202 to improve the performance of the second deep learning model finally obtained. Taking the optimization identifier as a quantization identifier as an example, in the above step 202, quantization processing can be further performed in the process of converting the first deep learning model to reduce the storage space occupied by the second deep learning model finally obtained; at this time, the model type of the second deep learning model finally obtained may be: a quantization type for indicating that the second deep learning model is a quantized model.

[0091] It should also be noted that in this embodiment, during the process of converting the first deep learning model in the above step 202, the first deep learning model can be further optimized based on the AI ​​device information of the target AI device to improve the computing performance of the second deep learning model obtained after the conversion. The optimization processing here may include: converting the topological graph of the first deep learning model into a structural graph recognizable by the target AI device, optimizing the computational graph of the first deep learning model, converting the data dimension format of the first deep learning model (the weight data format of the second deep learning model obtained after the conversion is nchw, nhwc, nchw_vec_c, etc.), offline generation of computing instructions, and other optimization processing.

[0092] Based on this, the model parameters of the second deep learning model in the above step 203 may include: the corresponding parameters after the above optimization process. For example: the model weight data format corresponding to the second deep learning model; the model type of the second deep learning model, the model type at least including: the quantization type used to indicate that the second deep learning model is a quantized model; when the model type of the second deep learning model is a quantized type, the quantization parameter corresponding to the second deep learning model. Optionally, the quantization parameter here may include: the number of quantization bits (such as 8 bits, 4 bits, 1 bit, etc.), the quantization algorithm, etc., which are not specifically limited in this embodiment.

[0093] The following describes an example of how to generate a target deep learning model of the first deep learning model based on the second deep learning model and the model description information used to describe the second deep learning model in step 203:

[0094] See also Figure 3 , Figure 3 The step 203 provided in the embodiment of the present application is implemented as a flow chart. Figure 3 As shown, the process may include the following steps:

[0095] Step 301: encapsulate the above model description information according to a specified model management protocol to obtain model encapsulation information.

[0096] Here, the model management protocol can be specified according to actual needs and is not specifically limited in this embodiment.

[0097] To facilitate tracing the target deep learning model, as an embodiment, this step 301 may further encapsulate the above-mentioned model conversion information. Based on this, this step 301 may include: encapsulating the above-mentioned model description information and the above-mentioned model conversion information according to the specified model management protocol to obtain the above-mentioned model encapsulation information. Of course, to save encapsulation resources, this step 301 may also exclude the first deep model in the model conversion information during encapsulation. That is, step 301 may include: encapsulating the model description information and the information in the model conversion information except the first deep model according to the specified model management protocol to obtain the above-mentioned model encapsulation information.

[0098] Step 302: Load the model packaging information into a specified location of the second deep learning model.

[0099] In one example, the designated position here may be the model protocol head position of the second deep learning model, or the end position of the second deep learning model, etc., which is not specifically limited in this embodiment.

[0100] As an extension, the model encapsulation information here can be expressed through json or xml.

[0101] Step 303: Package the second deep learning model with the model encapsulation information loaded at the designated location to obtain the target deep learning model.

[0102] Finally, through Figure 3 The process shown realizes the generation of a target deep learning model of the first deep learning model based on the second deep learning model and the model description information used to describe the second deep learning model.

[0103] It should be noted that, in one example, after generating the target deep learning model, to improve the security of the target deep learning model, the target deep learning model may be encrypted according to a predetermined encryption algorithm and stored. The predetermined encryption algorithm may be set based on actual needs, such as symmetric encryption, asymmetric encryption, one-way encryption, MD5, etc., and is not specifically limited in this embodiment.

[0104] Later, when needed, the user can export the stored encrypted target deep learning model and decrypt it with the help of a decryption tool. Based on the decrypted target deep learning model, the user can view the model description information and determine whether to deploy or upgrade the target deep learning model based on the viewed information, although this embodiment does not specifically limit this.

[0105] This completes the description of the method provided in this embodiment.

[0106] The following describes the system and device provided in this embodiment:

[0107] See also Figure 4 , Figure 4 This is the system structure diagram provided in this embodiment. Figure 4 As shown, the system may include: a client 401, a network model management server 402, and N AI devices 403, where N is greater than or equal to 1.

[0108] In one example, client 401 is client-facing, and a user controls client 401 to send model conversion information of the first deep learning model to be converted to network model management server 402, so that network model management server 402 obtains the model conversion information of the first deep learning model to be converted. The model conversion information here is as described above and will not be repeated here.

[0109] In one example, the network model management server 402 is connected between the client 401 and the AI ​​device 403. Alternatively, the client 401 and the AI ​​device 403 may be connected to the network model management server 102 via a network.

[0110] In one example, AI device 403 may be an AI device used in different application scenarios, such as the cloud, device, and edge. Alternatively, an AI device used in the cloud may be referred to as a cloud AI device. An AI device used on the device may be referred to as a device-side AI device. An AI device used on the network edge may be referred to as a network edge AI device.

[0111] The network model management server 402 may execute Figure 2 The process shown is specifically as follows: obtaining model conversion information of the first deep learning model to be converted from the client 401, the model conversion information including at least the first deep learning model and AI device information of the target AI device; the target AI device is an AI device among N AI devices on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device; and

[0112] The first deep learning model is converted to obtain a second deep learning model that matches the AI ​​device information of the target AI device; and a target deep learning model of the first deep learning model is generated based on the second deep learning model and the model description information used to describe the second deep learning model.

[0113] Optionally, the model description information includes at least: software version information for running the second deep learning model for deep learning reasoning and / or model parameters of the second deep learning model.

[0114] In one example, the network model management server 402 converting the first deep learning model may include: forwarding the model conversion information to the target AI device corresponding to the AI ​​device information, so that the target AI device converts the first deep learning model through a configured model conversion tool.

[0115] In one example, the network model management server 402 generates a target deep learning model of the first deep learning model based on the second deep learning model and the model description information used to describe the second deep learning model, including:

[0116] Encapsulating the model description information according to a specified model management protocol to obtain model encapsulation information;

[0117] Loading the model encapsulation information into a specified location of the second deep learning model;

[0118] The second deep learning model with the model encapsulation information loaded at the specified location is packaged to obtain the target deep learning model.

[0119] In one example, the network model management server 402 encapsulates the model description information according to a specified model management protocol to obtain model encapsulation information, including:

[0120] Encapsulating the model description information and the model conversion information according to a specified model management protocol to obtain the model encapsulation information; or,

[0121] The model description information and information in the model conversion information except the first depth model are encapsulated according to a specified model management protocol to obtain the model encapsulation information.

[0122] In one example, the software version information includes at least one of the following information:

[0123] The minimum version information of the inference software stack used to perform deep learning reasoning using the second deep learning model, and the type of the inference software stack used to perform deep learning reasoning using the second deep learning model.

[0124] In one example, the model parameters include at least one of the following parameters:

[0125] The model weight data format corresponding to the second deep learning model;

[0126] The model type of the second deep learning model; the model type includes at least: a quantization type for indicating that the second deep learning model is a quantized model;

[0127] When the model type of the second deep learning model is a quantization type, the quantization parameter corresponding to the second deep learning model.

[0128] In one example, the network model management server 402 may further include:

[0129] The target deep learning model is encrypted according to a set encryption algorithm and the encrypted target deep learning model is stored.

[0130] The above describes the system provided by the embodiment of the present application. The following describes the device provided by the embodiment of the present application:

[0131] See also Figure 5 , Figure 5 This is a diagram of the device structure provided in an embodiment of the present application. The device is applied to a network model management server and may include:

[0132] an acquiring unit, configured to acquire model conversion information of a first deep learning model to be converted, the model conversion information including at least: the first deep learning model and AI device information of a target AI device; the target AI device is an AI device on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device;

[0133] A first processing unit, configured to convert the first deep learning model to obtain a second deep learning model that matches the target AI device;

[0134] A second processing unit is configured to generate a target deep learning model of the first deep learning model based on the second deep learning model and model description information used to describe the second deep learning model.

[0135] Optionally, the model description information includes at least: software version information used to run the second deep learning model for deep learning reasoning and / or model parameters of the second deep learning model.

[0136] As an embodiment, the first processing unit converting the first deep learning model includes:

[0137] The model conversion information is forwarded to the target AI device corresponding to the AI ​​device information, so that the target AI device converts the first deep learning model through the configured model conversion tool.

[0138] As an embodiment, the second processing unit generating a target deep learning model of the first deep learning model according to the second deep learning model and the model description information used to describe the second deep learning model includes:

[0139] Encapsulating the model description information according to a specified model management protocol to obtain model encapsulation information;

[0140] Loading the model encapsulation information into a specified location of the second deep learning model;

[0141] The second deep learning model with the model encapsulation information loaded at the specified location is packaged to obtain the target deep learning model.

[0142] Optionally, the second processing unit encapsulates the model description information according to a specified model management protocol to obtain model encapsulation information, including:

[0143] Encapsulating the model description information and the model conversion information according to a specified model management protocol to obtain the model encapsulation information; or,

[0144] The model description information and information in the model conversion information except the first depth model are encapsulated according to a specified model management protocol to obtain the model encapsulation information.

[0145] As an embodiment, the software version information includes at least one of the following information:

[0146] The minimum version information of the inference software stack used to perform deep learning reasoning using the second deep learning model, and the type of the inference software stack used to perform deep learning reasoning using the second deep learning model.

[0147] As an embodiment, the model parameters include at least one of the following parameters:

[0148] The model weight data format corresponding to the second deep learning model;

[0149] The model type of the second deep learning model; the model type includes at least: a quantization type for indicating that the second deep learning model is a quantized model;

[0150] When the model type of the second deep learning model is a quantization type, the quantization parameter corresponding to the second deep learning model.

[0151] Optionally, the second processing unit may further encrypt the target deep learning model according to a set encryption algorithm and store the encrypted target deep learning model.

[0152] So far, completed Figure 5 Structural description of the device shown.

[0153] Correspondingly, this application also provides Figure 5 The hardware structure of the device shown. Figure 6 The hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0154] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0155] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0156] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0157] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0158] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0159] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0162] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A management method for model conversion, characterized in that: The method includes: Obtaining model conversion information of a first deep learning model to be converted, the model conversion information including at least: the first deep learning model and AI device information of a target AI device; the target AI device is an AI device on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device; Converting the first deep learning model to obtain a second deep learning model that matches the AI ​​device information of the target AI device; The model description information is encapsulated according to the specified model management protocol to obtain model encapsulation information; the model encapsulation information is loaded at the specified location of the second deep learning model; the second deep learning model with the model encapsulation information loaded at the specified location is packaged to obtain a target deep learning model; the model description information includes: software version information for running the second deep learning model for deep learning reasoning and / or model parameters of the second deep learning model; the model parameters include at least one of the following parameters: the model weight data format corresponding to the second deep learning model; the model type of the second deep learning model, the model type at least including: a quantization type for indicating that the second deep learning model is a quantized model; when the model type of the second deep learning model is a quantized type, the quantization parameters corresponding to the second deep learning model.

2. The method according to claim 1, characterized in that The converting of the first deep learning model includes: The model conversion information is forwarded to the target AI device corresponding to the AI ​​device information, so that the target AI device converts the first deep learning model through the configured model conversion tool.

3. The method according to claim 1, characterized in that The encapsulating the model description information according to the specified model management protocol to obtain the model encapsulation information includes: Encapsulating the model description information and the model conversion information according to a specified model management protocol to obtain the model encapsulation information; or, The model description information and information in the model conversion information except the first depth model are encapsulated according to a specified model management protocol to obtain the model encapsulation information.

4. The method according to claim 1, wherein The software version information includes at least one of the following information: The minimum version information of the inference software stack used to perform deep learning reasoning using the second deep learning model, and the type of the inference software stack used to perform deep learning reasoning using the second deep learning model.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The target deep learning model is encrypted according to a set encryption algorithm and the encrypted target deep learning model is stored.

6. A management system for model conversion, characterized in that: The system includes: A client, a network model management server, and N AI devices; wherein the N AI devices include at least one cloud-side AI device, and / or at least one terminal-side AI device, and / or at least one network edge-side AI device; N is greater than or equal to 1; the client and the N AI devices are respectively connected to the network model management server via a network; The network model management server obtains, from the client, model conversion information of a first deep learning model to be converted, the model conversion information including at least the first deep learning model and AI device information of a target AI device; the target AI device is an AI device among the N AI devices on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device; and The first deep learning model is converted to obtain a second deep learning model that matches the AI ​​device information of the target AI device, and the model description information is encapsulated according to the specified model management protocol to obtain model encapsulation information; the model encapsulation information is loaded at a specified location of the second deep learning model; the second deep learning model with the model encapsulation information loaded at the specified location is packaged to obtain a target deep learning model; the model description information includes: software version information for running the second deep learning model for deep learning inference and / or model parameters of the second deep learning model; the model parameters include at least one of the following parameters: the model weight data format corresponding to the second deep learning model; the model type of the second deep learning model, the model type including at least: a quantization type for indicating that the second deep learning model is a quantized model; when the model type of the second deep learning model is a quantized type, the quantization parameters corresponding to the second deep learning model.

7. A management device for model conversion, characterized in that: The device is applied to a network model management server and includes: an acquiring unit, configured to acquire model conversion information of a first deep learning model to be converted, the model conversion information including at least: the first deep learning model and AI device information of a target AI device; the target AI device is an AI device on which the first deep learning model is to be deployed, and the first deep learning model does not match the target AI device; A first processing unit, configured to convert the first deep learning model to obtain a second deep learning model that matches the target AI device; A second processing unit is configured to encapsulate the model description information in accordance with a specified model management protocol to obtain model encapsulation information; load the model encapsulation information at a specified location of the second deep learning model; and package the second deep learning model with the model encapsulation information loaded at the specified location to obtain a target deep learning model; the model description information includes: software version information for running the second deep learning model for deep learning reasoning and / or model parameters of the second deep learning model; the model parameters include at least one of the following parameters: a model weight data format corresponding to the second deep learning model; a model type of the second deep learning model, the model type including at least: a quantization type for indicating that the second deep learning model is a quantized model; and when the model type of the second deep learning model is a quantized type, the quantization parameters corresponding to the second deep learning model.

8. An electronic device, characterized in that: The electronic device includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method steps of any one of claims 1-5.

Images