Method, device, system, storage medium and electronic device for converting network model

Abstract

Embodiments of the present application provide a network model conversion method, device, system, storage medium and electronic device, wherein the method comprises: converting a first model file of a trained network model obtained by training in a first processor into an intermediate model file in a predetermined format; determining a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are different types of processors; determining an inference operator package based on the intermediate model file and the second model file; and sending the inference operator package to a first device comprising the second processor to instruct the first device to convert the trained network model into a target network model matching the second processor based on the inference operator package. Through the present application, the problem of high conversion cost of network models in different types of processors in the related art is solved, achieving the effect of reducing the conversion cost of network models in different types of processors.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of communications, and in particular, to a network model conversion method, device, system, storage medium, and electronic device. Background Art

[0002] In related technologies, deep learning models trained on GPU cards require model conversion before deployment on the inference side, requiring model adaptation based on different hardware platforms. During model conversion, the hardware environment on the inference side is also required to complete the final model conversion. For example, the GPU card used for training is an NVIDIA A card, and the GPU card for inference is an NVIDIA B card. To speed up inference, NVIDIA A uses TensorRT for acceleration. Therefore, the models used on the inference side are all TensorRT-accelerated models. Due to the different cores of NVIDIA A and NVIDIA B cards, models trained on NVIDIA A cards cannot be directly accelerated using TensorRT on NVIDIA B cards. Quantization and model conversion must be performed on an environment equipped with an NVIDIA B card. For example, configuring a separate hardware server with an NVIDIA B card to obtain a TensorRT-accelerated model that can be used on the NVIDIA B card increases user costs.

[0003] It can be seen from this that the related art has the problem of high cost in converting network models between different types of processors.

[0004] With respect to the above-mentioned problems existing in the related technologies, no effective solutions have been proposed so far. Summary of the Invention

[0005] Embodiments of the present invention provide a network model conversion method, apparatus, system, storage medium, and electronic device to at least solve the problem of high cost of converting network models between different types of processors in the related art.

[0006] According to one embodiment of the present invention, a method for converting a network model is provided, comprising: converting a first model file of a training network model obtained by training in a first processor into an intermediate model file in a predetermined format; determining a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are processors of different types; determining an inference operator package based on the intermediate model file and the second model file; and sending the inference operator package to a first device including the second processor to instruct the first device to convert the training network model into a target network model matching the second processor based on the inference operator package.

[0007] According to one embodiment of the present invention, a network model conversion method is also provided, including: sending a request message to a second device including a first processor, wherein the request message is used to request an inference operator package, the inference operator package is determined by the first processor based on an intermediate model file and a second model file corresponding to a model conversion module corresponding to the second processor included in the first device, and the intermediate model file is a file in a predetermined format converted from a first model file of a training network model trained in the first processor; based on the inference operator package, the training network model is converted into a target network model that matches the second processor.

[0008] According to one embodiment of the present invention, a network model conversion method is also provided, including: a second device sends an inference operator package to a first device, wherein the inference operator package is an operator package determined by the second device based on an intermediate model file and a second model file of a model conversion module corresponding to the first device, and the intermediate model file is a file in a predetermined format converted from a first model file of a training network model trained in the second device; the first device receives the inference operator package and converts the training network model into a target network model matching the first device based on the inference operator package, wherein the first device and the second device include different types of processors.

[0009] According to another embodiment of the present invention, a network model conversion device is provided, including: a first conversion module, used to convert a first model file of a training network model trained in a first processor into an intermediate model file in a predetermined format; a first determination module, used to determine a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are processors of different types; a second determination module, used to determine an inference operator package based on the intermediate model file and the second model file; and a second conversion module, used to send the inference operator package to a first device including the second processor, to instruct the first device to convert the training network model into a target network model matching the second processor based on the inference operator package.

[0010] According to another embodiment of the present invention, a network model conversion device is also provided, including: a sending module for sending request information to a second device including a first processor, wherein the request information is used to request an inference operator package, and the inference operator package is determined by the first processor based on an intermediate model file and a second model file of a model conversion module corresponding to the second processor included in the first device, and the intermediate model file is a file in a predetermined format converted from the first model file of a training network model trained in the first processor; a third conversion module for converting the training network model into a target network model matching the second processor based on the inference operator package.

[0011] According to another embodiment of the present invention, a network model conversion system is also provided, including: a first device, wherein the first device is used to execute the method as described in the above embodiment; a second device, wherein the second device is communicatively connected to the first device, and the second device is used to execute the method as described in the above embodiment, wherein the first device and the second device include different types of processors.

[0012] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0013] According to another embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0014] Through the present invention, the first model file of the training network model obtained by training in the first processor is converted into an intermediate model file of a predetermined format, the second model file of the model module corresponding to the second processor is determined, the inference operator package is determined based on the intermediate model file and the second model file, and the inference operator package is sent to the first device including the second processor to instruct the first device to convert the training network model into a target network model that matches the second processor based on the inference operator package. Since the first processor and the second processor are of different types, the first model file obtained by training in the first processor can be converted into an intermediate model file and the second model file of the model conversion module corresponding to the second processor can be determined. The first device can convert the training network model into a target network model based on the inference operator package determined by the intermediate model file and the second model file, so that the target network model matches the second processor. Moreover, the second model file is configured on the first processor side, and the second processor side does not need to configure a hardware server. Therefore, the problem of high cost of converting network models between different types of processors existing in the related art can be solved, and the cost of converting network models between different types of processors can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a hardware structure block diagram of a mobile terminal for a network model conversion method according to an embodiment of the present invention;

[0016] Figure 2 The process of the network model conversion method according to the embodiment of the present invention is as follows Figure 1 ;

[0017] Figure 3 The process of the network model conversion method according to the embodiment of the present invention is as follows Figure 2 ;

[0018] Figure 4 The process of the network model conversion method according to the embodiment of the present invention is as follows Figure 3 ;

[0019] Figure 5 1 is a schematic diagram of the structure of an apparatus for converting a running network model according to a specific embodiment of the present invention;

[0020] Figure 6 is a flow chart of determining an inference operator package according to a specific embodiment of the present invention;

[0021] Figure 7 is a flow chart of a method for converting a network model according to a specific embodiment of the present invention;

[0022] Figure 8 is a structural frame of a network model conversion device according to an embodiment of the present invention Figure 1 ;

[0023] Figure 9 is a structural frame of a network model conversion device according to an embodiment of the present invention Figure 2 . DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with embodiments.

[0025] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0026] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure diagram of a mobile terminal according to a method for converting a network model in an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a processing device such as a graphics processor GPU, a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. The mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the network model conversion method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] In this embodiment, a method for converting a network model is provided. Figure 2 The process of the network model conversion method according to the embodiment of the present invention is as follows Figure 1 ,like Figure 2 As shown, the process includes the following steps:

[0030] Step S202, converting the first model file of the training network model obtained by training in the first processor into an intermediate model file in a predetermined format;

[0031] Step S204, determining a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are different types of processors;

[0032] Step S206: determining an inference operator package based on the intermediate model file and the second model file;

[0033] Step S208: Send the inference operator package to the first device including the second processor to instruct the first device to convert the training network model into a target network model that matches the second processor based on the inference operator package.

[0034] In the above embodiment, the first processor and the second processor are different types of GPU processors, the computing power of the first processor is greater than that of the second processor, the first processor can be a processor in a training server, and the second processor can be a processor in an inference server, that is, the first device can be an inference server. The network model can be trained by the first processor. After obtaining the trained network model, the trained network model can be sent to a different first device including the second processor. Since the second processor and the first processor are different types of processors, it is necessary to convert the trained network model into a target network model that matches the second processor. When converting the target network model, the device including the first processor can convert the first model file of the trained network model into an intermediate model file in a predetermined format. It also determines the second model file of the model conversion module corresponding to the second processor, determines the inference operator package based on the intermediate model file and the second model file, and sends the inference operator package to the first device. After receiving the inference operator package, the first device can parse the inference operator package and implement model conversion based on the inference operator package. The predetermined format can be ONNX format.

[0035] In the above embodiment, the second model file is a file configured on the first processor side. When the second processor requests the network model, the device including the first processor can send the second model file and the intermediate model file to the second processor in the form of an inference operator package. There is no need to configure a hardware server for model conversion in the second processor, thereby achieving the effect of reducing costs.

[0036] Optionally, the execution entity of the above steps may be a server, such as a training server including a first processor, etc., but is not limited thereto.

[0037] Through the present invention, the first model file of the training network model obtained by training in the first processor is converted into an intermediate model file of a predetermined format, the second model file of the model conversion module corresponding to the second processor is determined, the inference operator package is determined based on the intermediate model file and the second model file, and the inference operator package is sent to the first device including the second processor to instruct the first device to convert the training network model into a target network model that matches the second processor based on the inference operator package. Since the first processor and the second processor are of different types, the first model file obtained by training in the first processor can be converted into an intermediate model file, and the second model file of the model conversion module corresponding to the second processor is determined, the first device can convert the training network model into a target network model based on the inference operator package determined by the intermediate model file and the second model file, so that the target network model matches the second processor. Moreover, the second model file is configured on the first processor side, and the second processor side does not need to configure a hardware server. Therefore, the problem of high cost of converting network models between different types of processors existing in the related art can be solved, and the cost of converting network models between different types of processors can be reduced.

[0038] In an exemplary embodiment, determining the inference operator package based on the intermediate model file and the second model file includes: obtaining the training data used to train the training network model; determining the inference algorithm corresponding to the second processor; and packaging the intermediate model file, the second model file, the training data, and the inference algorithm according to predetermined rules to obtain the inference operator package. In this embodiment, when determining the inference operator package, the training data can be obtained, the inference algorithm corresponding to the second processor can be determined, and the intermediate model file, the second model file, the training data, and the inference algorithm can be packaged according to predetermined rules to obtain the inference operator package. The second model file can be a data file corresponding to the model conversion module, and the second processor can run the second model file to convert the intermediate model file. The training data can be all the training data of the training network model or part of the training data of the training network model. The training data is used in the second processor to verify whether the converted target network model is normal.

[0039] In the above embodiment, the inference algorithm can be an algorithm used to start the target network model. The inference algorithm corresponds to the type of the second processor. After obtaining the target network model in the second processor, the inference algorithm can be loaded to start the target network model to perform inference operations.

[0040] In an exemplary embodiment, after determining the inference operator package based on the intermediate model file and the second model file, the method further includes: determining a uniform resource locator (URL) of the inference operator package; and sending the uniform resource locator (URL) to the first device to instruct the first device to send a request message based on the uniform resource locator (URL), wherein the request message is used to request the inference operator package. In this embodiment, after determining the inference operator package in the first processor, the uniform resource locator (URL) of the inference operator package can be determined, and the URL can be sent to the first device to instruct the first device to send a request message requesting the inference operator package according to the URL. The first device can be an inference server, an inference device, etc.

[0041] In an exemplary embodiment, after sending the uniform resource locator to the first device, the method further includes: receiving the request information sent by the first device; and sending the inference operator package to the first device based on the request information. In this embodiment, after receiving the URL, the second processor can send the request information to the first processor based on the URL, and the first processor can send the inference operator package to the first device based on the request information. In other words, the first device can download the inference operator package based on the request information.

[0042] In an exemplary embodiment, after determining the inference operator package based on the intermediate model file and the second model file, the method further includes: sending the inference operator package to the target second processor that is communicatively connected to the first processor according to a predetermined time period. In this embodiment, after the inference operator package is determined in the first processor, the inference operator package can be directly sent to the target second processor, wherein the target second processor can be one or more, and the target second processor is communicatively connected to the first processor. When sending the inference operator package, the first processor can send the inference operator package to the online target second processor. In order to prevent sending failure due to communication failure, or receiving failure due to some target second processors being offline, the inference operator package can be sent according to a predetermined time period.

[0043] In an exemplary embodiment, before sending the inference operator packet to a target second processor communicatively connected to the first processor according to a predetermined time period, the method further includes: receiving a response message, determining a third processor that received the inference operator packet based on the response message, determining a second processor communicatively connected to the first processor, and determining a processor in the second processor other than the third processor as the target second processor, wherein the response message is sent by the processor after receiving the inference operator packet. In this embodiment, before each inference operator packet is sent, the first processor can determine which processors to send the inference operator packet to avoid repeatedly sending the inference operator packet to a processor that has already received the inference operator packet. The first processor can determine the third processor that has already received the inference operator packet based on the received response message. Furthermore, the first processor can determine a second processor connected to the first processor, and the processors in the second processor other than the third processor are processors that have not received the inference operator packet. Therefore, the processors in the second processor other than the third processor can be determined as the target second processor.

[0044] In this embodiment, a method for converting a network model is provided. Figure 3 The process of the network model conversion method according to the embodiment of the present invention is as follows Figure 2 ,like Figure 3 As shown, the process includes the following steps:

[0045] Step S302: Send a request message to a second device including a first processor, wherein the request message is used to request an inference operator package, the inference operator package being determined by the first processor based on an intermediate model file and a second model file of a model conversion module corresponding to the second processor included in the first device, the intermediate model file being a file in a predetermined format converted from the first model file of a training network model trained in the first processor;

[0046] Step S304: Convert the training network model into a target network model that matches the second processor based on the inference operator package.

[0047] In the above embodiment, the first processor and the second processor are different types of GPU processors, and the computing power of the first processor is greater than that of the second processor. The second device can be a processor in a training server, and the second processor can be a processor in an inference server, that is, the first device can be an inference server. The network model can be trained by the first processor. After obtaining the trained network model, the trained network model can be sent to different inference devices including the second processor. Since the second processor and the first processor are different types of processors, the trained network model needs to be converted into a target network model that matches the second processor. When converting the target network model, the first model file of the training network model can be converted into an intermediate model file in a predetermined format in the second device. The second model file of the model conversion module corresponding to the second processor is determined, and an inference operator package is determined based on the intermediate model file and the second model file. The inference operator package is sent to the first device. After receiving the inference operator package, the first device can parse the inference operator package and implement model conversion based on the inference operator package. The predetermined format can be ONNX format.

[0048] In the above embodiment, the second model file is a file configured on the first processor side. When the second processor requests the network model, the second device sends the second model file and the intermediate model file to the first device in the form of an inference operator package. There is no need to configure a hardware server for model conversion in the first device, thereby achieving the effect of reducing costs.

[0049] Optionally, the execution entity of the above steps may be the first device, but is not limited thereto.

[0050] Through the present invention, the second device converts the first model file of the training network model trained in the first processor into an intermediate model file in a predetermined format, determines the second model file of the model conversion module corresponding to the second processor, and determines the inference operator package based on the intermediate model file and the second model file. The first device sends a request message for requesting the inference operator package to the first processor included in the second device, and sends the inference operator package to the second device, instructing the second device to convert the training network model into a target network model that matches the second processor based on the inference operator package. Since the first processor and the second processor can be of different types, the first model file trained in the first processor can be converted into an intermediate model file and the second model file of the model conversion module corresponding to the second processor can be determined. The second processor can convert the training network model into a target network model based on the intermediate model file and the inference operator package determined by the second model file, so that the target network model matches the second processor. The second model file is configured on the first processor side, and the second processor side does not need to configure a hardware server. Therefore, the problem of high cost of converting network models between different types of processors in the related art can be solved, achieving the effect of reducing the cost of converting network models between different types of processors.

[0051] In an exemplary embodiment, converting the training network model into a target network model that matches the second processor based on the inference operator package includes: parsing the inference operator package to obtain an intermediate model file, the second model file, and training data; running the second model file to obtain a model conversion module; and performing model conversion based on the model conversion module, the intermediate model file, and the training data to obtain the target network model. In this embodiment, the inference operator package can be an operator package packaged according to predetermined rules, and the second processor can parse it according to predetermined rules to obtain an intermediate model file, a second model file, and training data. The second model file can be a data file corresponding to the model conversion module, and the second processor can run the second model file to convert the intermediate model file. The training data can be all the training data of the training network model, or it can be part of the training data of the training network model. The training data is used in the second processor to verify whether the converted target network model is normal.

[0052] In the above embodiment, the inference algorithm can be an algorithm used to start the target network model. The inference algorithm corresponds to the type of the second processor. After obtaining the target network model in the second processor, the inference algorithm can be loaded to start the target network model to perform inference operations.

[0053] In an exemplary embodiment, performing model conversion based on the model conversion module, the intermediate model file, and the training data to obtain the target network model includes: loading the intermediate model file through the model conversion module to obtain the converted network model; inputting the training samples included in the training data into the conversion network model to determine the training result; determining the error between the label result corresponding to the training sample and the training result; and determining the conversion network model as the target network model when the error is less than a predetermined threshold. In this embodiment, when performing model conversion, the conversion network model can be obtained by loading the intermediate model file through the model conversion module, and the conversion network model can be verified based on the training data to determine the training result. When the error between the training result and the label result corresponding to the training sample is less than a predetermined threshold, the conversion network model is determined as the target network model.

[0054] In an exemplary embodiment, after determining the conversion network model as the target network model, the method further includes: determining the inference algorithm included in the inference operator package, wherein the inference algorithm is the inference algorithm corresponding to the second processor; running the inference algorithm to start the target network model; and performing a target operation based on the started target network model. In this embodiment, after obtaining the target network model, the inference algorithm included in the inference operator package can also be determined, and the target network model can be started by running the inference algorithm. The inference algorithm can be an algorithm corresponding to the second processor. After starting, the target network model can perform the corresponding target operation. For example, when the target network model is a face recognition model, the target operation can be a face recognition operation. When the target network model is a vehicle recognition model, the target operation can be a vehicle recognition operation.

[0055] In an exemplary embodiment, before sending the request information to the first processor, the method further includes: receiving a uniform resource locator (URL) of the inference operator package sent by the second device; and sending the request information based on the uniform resource locator (URL). In this embodiment, before sending the request information to the second device, a uniform resource locator (URL) of the inference operator package sent by the second device can be received, and the request information can be sent based on the URL to download the inference operator package on the second device.

[0056] In an exemplary embodiment, sending the request information to the first processor includes: determining the existence status of the target network model; and sending the request information to the second device if the existence status indicates that the target network model does not exist. In this embodiment, before sending the request information to the second device, the existence status of the target network model in the first device can be determined. If the target network model exists in the first device, there is no need to send the request information. If the target network model does not exist in the first device, the request information is sent.

[0057] In an exemplary embodiment, after converting the trained network model into a target network model that matches the second processor based on the inference operator packet, the method further includes: sending a response message to the first processor, wherein the response message is a message sent by the second processor after receiving the inference operator packet. In this embodiment, after receiving the inference operator packet, the second processor can send a response message to the first processor, and the first processor can determine that the second processor has received the inference operator packet based on the response message.

[0058] This embodiment also provides a network model conversion method. Figure 4 The process of the network model conversion method according to the embodiment of the present invention is as follows Figure 3 ,like Figure 4 As shown, the process includes the following steps:

[0059] Step S402: The second device sends an inference operator package to the first device, wherein the inference operator package is an operator package determined by the second device based on an intermediate model file and a second model file of a model conversion module corresponding to the first device, and the intermediate model file is a file in a predetermined format converted from the first model file of a training network model trained in the first device;

[0060] In step S404, the first device receives the inference operator package and converts the training network model into a target network model that matches the first device based on the inference operator package, wherein the first device and the second device include different types of processors.

[0061] In the above embodiment, the first device may include a second processor, and the second device may include a first processor. The first and second processors may be different types of GPU processors. The computing power of the first processor is greater than that of the second processor. The first processor may be a processor in a training server, and the second processor may be a processor in an inference server, i.e., the first device may be an inference server. A network model may be trained using the first processor. After obtaining the trained network model, the trained network model may be sent to a different first device including a second processor. Since the second processor and the first processor are different types, the trained network model needs to be converted into a target network model that matches the second processor. When converting the target network model, the device including the first processor may convert the first model file of the trained network model into an intermediate model file in a predetermined format. The device may also determine a second model file for a model conversion module corresponding to the second processor, determine an inference operator package based on the intermediate model file and the second model file, and send the inference operator package to the first device. Upon receiving the inference operator package, the first device may parse the inference operator package and perform model conversion based on the inference operator package. The predetermined format may be ONNX format.

[0062] In the above embodiment, the second model file is a file configured on the first processor side. When the second processor requests the network model, the device including the first processor can send the second model file and the intermediate model file to the second processor in the form of an inference operator package. There is no need to configure a hardware server for model conversion in the second processor, thereby achieving the effect of reducing costs.

[0063] Through the present invention, the second device determines an inference operator package based on an intermediate model file in a predetermined format converted from the first model file of the training network model trained in the second device and the second model file of the model conversion module corresponding to the first device. The second device sends the inference operator package to the first device. After receiving the inference operator package, the first device converts the training network model into a target network model that matches the first device based on the inference operator package. Since the first and second devices can include different types of processors, the first model file trained in the second device can be converted into an intermediate model file and the second model file of the model conversion module corresponding to the first device can be determined. This allows the first device to convert the training network model into a target network model based on the intermediate model file and the inference operator package determined by the second model file, so that the target network model matches the first device. The second model file is configured on the second device side, and the first device side does not need to configure a hardware server. Therefore, the problem of high cost of converting network models between different types of processors existing in the related art can be solved, achieving the effect of reducing the cost of converting network models between different types of processors.

[0064] The following describes the network model conversion method in conjunction with specific implementation methods:

[0065] Figure 5 is a schematic diagram of the structure of a device for converting a running network model according to a specific embodiment of the present invention. Figure 5 As shown, the first device can be a device included in the inference unit, and the second device can be a training service unit. The training service unit can include a model training module, an intermediate model conversion module, an inference operator package packaging module, and a deployment module. The model training module is responsible for training the original model using the NVIDIA AGPU card; the intermediate model conversion module mainly converts the trained original model file into an intermediate model file in the standard ONNX format. After receiving the ONNX intermediate model output by the intermediate module, the inference operator packaging module, along with the adapted NVIDIA B model conversion module, the NVIDIA B inference algorithm package, and the calibration image, generates an inference operator package according to certain packaging rules. The deployment module completes the distribution of the generated inference operator package URL to each inference device. After receiving the inference operator package URL issued by the training service unit, the inference unit initiates a request to the training service unit to download the inference operator package. After the training service unit verifies and passes the inference operator package, the inference operator package is obtained.

[0066] Figure 6 is a flow chart of determining an inference operator package according to a specific embodiment of the present invention, such as Figure 6 As shown in the figure, when the first processor is an NVIDIA A GPU card and the second processor is an NVIDIA B card, after receiving the ONNX intermediate model output by the intermediate module, an inference operator package is generated according to certain packaging rules together with the NVIDIA B card model conversion module, the NVIDIA B card inference algorithm package and the calibration image (corresponding to the above training data).

[0067] Figure 7 is a flow chart of a method for converting a network model according to a specific embodiment of the present invention, such as Figure 7 As shown, the process includes:

[0068] a) After the inference unit (corresponding to the first device mentioned above) is started, it first determines whether the TensorRT model based on the B inference card exists. If not, it first parses the inference operator package downloaded from the training service unit to obtain the intermediate ONNX model, model conversion module, verification image, and inference algorithm;

[0069] b) Start the model conversion module and input the intermediate ONNX model and verification image according to certain rules to perform model conversion based on the B inference card hardware environment;

[0070] c) After the model is converted to the TensorRT acceleration model, start the inference algorithm to load the TensorRT model file and perform inference operations.

[0071] d) If the TensorRT model already exists, directly load the TensorRT model file for inference. This eliminates the need to convert the TensorRT model again, speeding up the startup of the inference algorithm.

[0072] In the aforementioned embodiment, TensorRT acceleration is achieved by prepending the model conversion module to the inference side and invoking it to perform TensorRT model conversion when the inference side starts. The integration of the inference algorithm and the model conversion module eliminates the need to launch separate tools for model conversion, reducing user costs and complexity.

[0073] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0074] In this embodiment, a network model conversion device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. The details already described will not be repeated here. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0075] Figure 8 is a structural frame of a network model conversion device according to an embodiment of the present invention Figure 1 ,like Figure 8 As shown, the device includes:

[0076] A first conversion module 82 is configured to convert a first model file of the training network model obtained by training in the first processor into an intermediate model file in a predetermined format;

[0077] A first determining module 84 is configured to determine a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are different types of processors;

[0078] A second determining module 86, configured to determine an inference operator package based on the intermediate model file and the second model file;

[0079] The second conversion module 88 is used to send the inference operator package to the first device including the second processor to instruct the first device to convert the training network model into a target network model matching the second processor based on the inference operator package.

[0080] Among them, the first conversion module 82 corresponds to the above-mentioned intermediate model conversion module, the first determination module 84 and the second determination module 86 correspond to the above-mentioned inference operator package packaging module, and the second conversion module 88 corresponds to the deployment module.

[0081] In an exemplary embodiment, the second determination module 86 can determine the inference operator package based on the intermediate model file and the second model file in the following manner: obtain the training data used to train the training network model; determine the inference algorithm corresponding to the second processor; and package the intermediate model file, the second model file, the training data and the inference algorithm according to predetermined rules to obtain the inference operator package.

[0082] In an exemplary embodiment, the apparatus may be configured to determine a uniform resource locator (URL) of the inference operator package after determining the inference operator package based on the intermediate model file and the second model file; and send the URL to the first device to instruct the first device to send request information based on the URL, wherein the first device includes the second processor, and the request information is used to request the inference operator package.

[0083] In an exemplary embodiment, the apparatus may also be configured to, after sending the uniform resource locator to the first device, receive the request information sent by the first device; and send the inference operator package to the first device based on the request information.

[0084] In an exemplary embodiment, the apparatus may be further configured to, after determining the inference operator package based on the intermediate model file and the second model file, send the inference operator package to a target second processor communicatively connected to the first processor according to a predetermined time period.

[0085] In an exemplary embodiment, the device can also be used to receive a response message before sending the inference operator package to the target second processor that is communicatively connected to the first processor according to a predetermined time period, determine the third processor that has received the inference operator package based on the response message, determine the second processor that is communicatively connected to the first processor, and determine the processor in the second processor other than the third processor as the target second processor, wherein the response message is a message sent by the processor after receiving the inference operator package.

[0086] Figure 9 is a structural frame of a network model conversion device according to an embodiment of the present invention Figure 2 ,like Figure 9 As shown, the device includes:

[0087] a sending module 92 configured to send a request message to a second device including a first processor, wherein the request message is used to request an inference operator package, the inference operator package being determined by the first processor based on an intermediate model file and a second model file of a model conversion module corresponding to the second processor included in the first device, the intermediate model file being a file in a predetermined format converted from the first model file of a training network model trained in the first processor;

[0088] The third conversion module 94 is configured to convert the training network model into a target network model matching the second processor based on the inference operator package.

[0089] In an exemplary embodiment, the third conversion module 94 can convert the training network model into a target network model that matches the second processor based on the inference operator package in the following manner: parse the inference operator package to obtain an intermediate model file, the second model file, and training data; run the second model file to obtain a model conversion module; perform model conversion based on the model conversion module, the intermediate model file, and the training data to obtain the target network model.

[0090] In an exemplary embodiment, the third conversion module 94 can implement model conversion based on the model conversion module, the intermediate model file and the training data to obtain the target network model in the following manner: load the intermediate model file through the model conversion module to obtain the converted conversion network model; input the training samples included in the training data into the conversion network model to determine the training results; determine the error between the label results corresponding to the training samples and the training results; when the error is less than a predetermined threshold, determine the conversion network model as the target network model.

[0091] In an exemplary embodiment, the device can be used to determine the inference algorithm included in the inference operator package after determining the conversion network model as the target network model, wherein the inference algorithm is the inference algorithm corresponding to the second processor; run the inference algorithm to start the target network model; and perform the target operation based on the started target network model.

[0092] In an exemplary embodiment, the apparatus may be configured to receive a uniform resource locator (URL) of the inference operator package sent by the second device before sending the request information to the first processor; and send the request information based on the URL.

[0093] In an exemplary embodiment, the sending module 92 may send the request information to the first processor in the following manner: determining the existence status of the target network model; and sending the request information to the second device when the existence status indicates that the target network model does not exist.

[0094] In an exemplary embodiment, the device is also used to send a response message to the first processor after converting the training network model into a target network model that matches the second processor based on the inference operator package, wherein the response message is a message sent by the second processor after receiving the inference operator package.

[0095] In this embodiment, a network model conversion system is also provided, including a first device, wherein the first device is used to perform the method of converting the network model described in the above embodiment (i.e., the process of the network model conversion method Figure 1 The method in the corresponding embodiment); a second device, wherein the second device is connected to the first device for communication, and the second device is used to perform the method 2 of the conversion of the network model described in the above embodiment (i.e., the process of the conversion method of the network model Figure 2 Corresponding embodiment), wherein the first device and the second device include different types of processors.

[0096] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0097] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0098] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0099] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0100] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0101] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0102] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0103] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A network model conversion method, characterized in that: include: Converting a first model file of the training network model obtained by training in the first processor into an intermediate model file in a predetermined format; Determining a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are different types of processors; Determining an inference operator package based on the intermediate model file and the second model file, wherein the second model file is configured on the first processor side; Sending the inference operator packet to a first device including the second processor to instruct the first device to convert the training network model into a target network model matching the second processor based on the inference operator packet; Determining the inference operator package based on the intermediate model file and the second model file includes: obtaining the training data used to train the training network model; determining the inference algorithm corresponding to the second processor; and packaging the intermediate model file, the second model file, the training data and the inference algorithm according to predetermined rules to obtain the inference operator package.

2. The method according to claim 1, characterized in that After determining the inference operator package based on the intermediate model file and the second model file, the method further includes: Determining a uniform resource locator of the inference operator package; The uniform resource locator is sent to the first device to instruct the first device to send request information based on the uniform resource locator, wherein the request information is used to request the inference operator package.

3. The method according to claim 2, characterized in that After sending the uniform resource locator to the first device, the method further includes: receiving the request information sent by the first device; The inference operator packet is sent to the first device based on the request information.

4. A network model conversion method, characterized in that: include: Sending a request message to a second device including a first processor, wherein the request message is used to request an inference operator package, the inference operator package being determined by the first processor based on an intermediate model file and a second model file of a model conversion module corresponding to the second processor included in the first device, the intermediate model file being a file in a predetermined format converted from the first model file of a training network model trained in the first processor, and the second model file being configured on the first processor side; Converting the training network model into a target network model matching the second processor based on the inference operator package; The inference operator package is obtained by the first processor in the following manner: obtaining the training data used to train the training network model; determining the inference algorithm corresponding to the second processor; and packaging the intermediate model file, the second model file, the training data and the inference algorithm according to predetermined rules to obtain the inference operator package.

5. The method according to claim 4, characterized in that Converting the training network model into a target network model matching the second processor based on the inference operator package includes: Parse the inference operator package to obtain the intermediate model file, the second model file of the model conversion module, and training data. Running the second model file to obtain a model conversion module; Model conversion is performed based on the model conversion module, the intermediate model file and the training data to obtain the target network model.

6. The method according to claim 5, characterized in that Performing model conversion based on the model conversion module, the intermediate model file, and the training data to obtain the target network model includes: Loading the intermediate model file through the model conversion module to obtain a converted conversion network model; Inputting the training samples included in the training data into the conversion network model to determine the training results; Determining an error between a label result corresponding to the training sample and the training result; When the error is smaller than a predetermined threshold, the converted network model is determined as the target network model.

7. The method according to claim 6, characterized in that After determining the conversion network model as the target network model, the method further includes: Determining an inference algorithm included in the inference operator package, wherein the inference algorithm is an inference algorithm corresponding to the second processor; Running the inference algorithm to start the target network model; Target operations are performed based on the activated target network model.

8. The method according to claim 4, characterized in that Before sending the request information to the first processor, the method further includes: receiving a uniform resource locator of the inference operator packet sent by the second device; The request information is sent based on the uniform resource locator.

9. The method according to claim 4, characterized in that Sending the request information to the first processor includes: Determining the existence status of the target network model; In a case where the existence status indicates that the target network model does not exist, request information is sent to the second device.

10. A network model conversion method, characterized in that: include: The second device sends an inference operator package to the first device, wherein the inference operator package is an operator package determined by the second device based on an intermediate model file and a second model file of a model conversion module corresponding to the first device, the intermediate model file is a file in a predetermined format converted from the first model file of a training network model trained in the second device, the second device includes a first processor, and the second model file is configured on the first processor side; The first device receives the inference operator package and converts the training network model into a target network model that matches the first device based on the inference operator package, wherein the first device and the second device include different types of processors, and the first device includes the second processor; The inference operator package is obtained by the second device in the following manner: obtaining the training data used to train the training network model; determining the inference algorithm corresponding to the second processor; and packaging the intermediate model file, the second model file, the training data and the inference algorithm according to predetermined rules to obtain the inference operator package.

11. A network model conversion device, characterized in that: include: A first conversion module, configured to convert a first model file of the training network model obtained by training in the first processor into an intermediate model file in a predetermined format; A first determining module, configured to determine a second model file of a model conversion module corresponding to a second processor, wherein the second processor and the first processor are different types of processors; a second determining module, configured to determine an inference operator package based on the intermediate model file and the second model file, wherein the second model file is configured on the first processor side; a second conversion module, configured to send the inference operator packet to a first device including the second processor, to instruct the first device to convert the training network model into a target network model matching the second processor based on the inference operator packet; The second determination module determines the inference operator package based on the intermediate model file and the second model file in the following manner: obtaining the training data used to train the training network model; determining the inference algorithm corresponding to the second processor; and packaging the intermediate model file, the second model file, the training data and the inference algorithm according to predetermined rules to obtain the inference operator package.

12. A network model conversion device, characterized in that: include: a sending module, configured to send a request message to a second device including a first processor, wherein the request message is used to request an inference operator package, the inference operator package being determined by the first processor based on an intermediate model file and a second model file of a model conversion module corresponding to the second processor included in the first device, the intermediate model file being a file in a predetermined format converted from the first model file of a training network model trained in the first processor, and the second model file being configured on the first processor side; A third conversion module, configured to convert the training network model into a target network model matching the second processor based on the inference operator package; The inference operator package is obtained by the first processor in the following manner: obtaining the training data used to train the training network model; determining the inference algorithm corresponding to the second processor; and packaging the intermediate model file, the second model file, the training data and the inference algorithm according to predetermined rules to obtain the inference operator package.

13. A network model conversion system, characterized in that: include: A first device, wherein the first device is configured to perform the method according to any one of claims 4 to 9; A second device, wherein the second device is communicatively connected to the first device, and the second device is used to execute the method according to any one of claims 1 to 3, wherein the first device and the second device include different types of processors.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 3 are implemented, or the steps of the method described in any one of claims 4 to 10 are implemented.

15. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method described in any one of claims 1 to 3, or to execute the method described in any one of claims 4 to 10.

Images