Deep learning model transformation deployment methods, devices, storage media and electronic devices

Abstract

This disclosure relates to the field of artificial intelligence technology, specifically to a method, apparatus, storage medium, and electronic device for converting and deploying deep learning models. The method includes acquiring a model file to be processed obtained by training a model using a first deep learning framework; performing an upsampling operation on the model file using a deconvolution function to obtain an initial model file; performing data normalization processing on the initial model file using a normalization function to obtain an intermediate model file; and performing linear correction on the intermediate model file using an activation function to obtain a target model file suitable for processing in a second deep learning framework. The deep learning model conversion and deployment method provided by this disclosure can solve the technical problems of high development workload, poor accuracy, and high cost in existing technologies when converting PyTorch models to TensorRT models for deployment.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to a deep learning model conversion and deployment method, a deep learning model conversion and deployment device, a storage medium, and an electronic device. Background Technology

[0002] In engineering applications, PyTorch models can be converted to TensorRT engines to accelerate computation. However, when converting a PyTorch model to a TensorRT engine file (version 5.0.2.6), the upsampling operation `torch.nn.Upsample()` fails to convert the model to the TensorRT engine.

[0003] Among the existing technologies, one approach is to develop a customized plugin for TensorRT 5.0.2.6, but this method involves a large amount of engineering development and is quite cumbersome. Another approach is to first generate an intermediate transitional model ONNX from PTH, and then generate a TensorRT 5.0.2.6 model from ONNX. This method requires conversion to the ONNX intermediate model, resulting in two numerical accuracy errors. Alternatively, one approach is to upgrade TensorRT to version 7.0 or higher, but this method requires discarding TensorRT version 5.0.2.6, resulting in high software and firmware upgrade costs.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a method, apparatus, storage medium, and electronic device for converting and deploying deep learning models, aiming to solve the technical problems of large development workload, poor accuracy, and high cost in the existing technology when converting PyTorch models to TensorRT models for deployment.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of the present disclosure, a deep learning model conversion and deployment method is provided, comprising: obtaining a model file to be processed obtained by training a model using a first deep learning framework; calling a deconvolution function to perform an upsampling operation on the model file to be processed to obtain an initial model file; performing data normalization processing on the initial model file using a normalization function to obtain an intermediate model file; and using an activation function to perform linear correction on the intermediate model file to obtain a target model file, which is suitable for processing in a second deep learning framework.

[0008] According to some embodiments of this disclosure, based on the foregoing scheme, the first deep learning framework is a PyTorch model, and the model file to be processed is a pth file.

[0009] According to some embodiments of this disclosure, based on the foregoing scheme, the second deep learning framework is a TensorRT model of version 5.0.2.6, and the target model file is a TensorRT engine file.

[0010] According to some embodiments of this disclosure, based on the foregoing scheme, the deconvolution function is a ConvTranspose2d function, and the step of calling the deconvolution function to perform an upsampling operation on the model file to be processed includes: padding the feature map in the model file to be processed to obtain a new feature map; randomly initializing the convolution kernel of the target size; and calling the deconvolution function to perform a convolution operation on the new feature map using the convolution kernel.

[0011] According to some embodiments of this disclosure, based on the foregoing scheme, the normalization function is the BatchNorm2d function, and the step of calling the normalization function to perform data normalization processing on the initial model file includes: configuring the internal parameters of the normalization function; and performing data normalization processing on the feature data in the initial model file based on the internal parameters.

[0012] According to some embodiments of this disclosure, based on the foregoing scheme, the step of calling the activation function to perform linear correction on the intermediate model file includes: obtaining target feature data with negative data values ​​from the feature data in the intermediate model file; and changing the data values ​​of the target feature data based on the activation function.

[0013] According to some embodiments of this disclosure, based on the foregoing scheme, the activation function is a ReLU function or a Sigmoid function.

[0014] According to a second aspect of the present disclosure, a deep learning model conversion and deployment apparatus is provided, comprising: an acquisition module for acquiring a model file to be processed obtained by training a model using a first deep learning framework; a deconvolution module for calling a deconvolution function to perform an upsampling operation on the model file to be processed to obtain an initial model file; a normalization module for performing data normalization processing on the initial model file using a normalization function to obtain an intermediate model file; and an activation function module for calling an activation function to perform linear correction on the intermediate model file to obtain a target model file suitable for processing in a second deep learning framework.

[0015] According to a third aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the deep learning model conversion deployment method as described in the above embodiments.

[0016] According to a fourth aspect of the present disclosure, an electronic device is provided, characterized in that it includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the deep learning model conversion deployment method as described in the above embodiments.

[0017] The exemplary embodiments disclosed herein may have some or all of the following beneficial effects:

[0018] In the technical solutions provided by some embodiments of this disclosure, upsampling, normalization, and linear correction are sequentially performed on the model file to be processed obtained by training a first deep learning framework to finally obtain a target model file. This target model file can then be applied to processing in a second deep learning framework. This allows the upsampling operation to be replaced with an operation applicable to the second deep learning framework without changing the input (the model file to be processed) and output (the target model file) of the existing upsampling operation, making it suitable for processing in the second deep learning framework. On the one hand, this avoids the development of customized plugins suitable for the second deep learning framework, resulting in less development and testing, simpler engineering implementation, and better model performance. On the other hand, it avoids the introduction of intermediate transition models, preventing loss of accuracy and performance, and eliminating the space occupied by intermediate model storage. Furthermore, it eliminates the need for version upgrades, thus avoiding the software and hardware incompatibility and upgrade testing costs associated with upgrades.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0021] Figure 1 The illustration shows a flowchart of a deep learning model conversion deployment method in an exemplary embodiment of the present disclosure.

[0022] Figure 2 The schematic diagram illustrates a flowchart of an upsampling operation method in an exemplary embodiment of the present disclosure;

[0023] Figure 3 The illustration shows a flowchart of a data normalization processing method in an exemplary embodiment of the present disclosure.

[0024] Figure 4 The schematic diagram illustrates a flow chart of a linear correction method in an exemplary embodiment of the present disclosure;

[0025] Figure 5 This schematic diagram illustrates the composition of a deep learning model conversion deployment apparatus according to an exemplary embodiment of the present disclosure;

[0026] Figure 6 This schematic diagram illustrates a computer-readable storage medium according to an exemplary embodiment of the present disclosure;

[0027] Figure 7 The schematic diagram illustrates the structure of a computer system of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0029] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0032] In engineering applications, PyTorch models are converted into TensorRT engines, thereby accelerating computation. The deployment of a PyTorch deep learning model typically involves two steps: training the model using PyTorch to generate the pth model file; using the pth file to generate the TensorRT engine file; and finally, running the TensorRT engine file on an embedded device to achieve the desired acceleration.

[0033] When converting a PyTorch model to a TensorRT engine file (version 5.0.2.6), the upsampling operation `torch.nn.Upsample()` may fail to convert successfully. This is because TensorRT version 5.0.2.6 does not support the `torch.nn.Upsample()` operation. In this case, the `torch.nn.Upsample()` operation needs to be replaced with one supported by TensorRT 5.0.2.6. Currently, there are three main solutions:

[0034] 1. Develop a custom plugin for TensorRT 5.0.2.6. However, this method involves a large amount of engineering development and is quite cumbersome. It requires compiling the CUDA code and linking it to TensorRT; the custom plugin needs to implement the same functionality as torch.nn.Upsample(), and it needs to ensure that, under certain input conditions, the output of the custom plugin is strictly consistent with the output of torch.nn.Upsample(); the testing work required to ensure this consistency is substantial.

[0035] 2. Generate an intermediate ONNX model from the PTH and then convert it into a TensorRT engine file. However, this method requires conversion to an ONNX intermediate model, which consumes additional disk space. Furthermore, the conversion from PTH to ONNX introduces numerical errors, and the conversion from ONNX to TensorRT also introduces errors, resulting in two numerical precision errors. This causes the output model to perform worse than the original model. In addition, this method is more computationally intensive and time-consuming than directly generating a TensorRT engine file from PTH.

[0036] 3. Upgrade TensorRT to version 7.0 or higher. However, this method requires discarding TensorRT version 5.0.2.6. In engineering applications, version upgrades involve significant changes in various aspects, requiring models previously stably deployed using TensorRT 5.0.2.6 to be retested on version 7.0. Furthermore, existing software versions need to be upgraded. For hardware products already running TensorRT 5.0.2.6, without upgrading, the current PTH model cannot be deployed, leading to compatibility issues, and the software firmware upgrade is costly.

[0037] Therefore, addressing the numerous problems existing in deploying PyTorch-trained models using Tensor models in current technologies, this disclosure provides a deep learning model conversion and deployment method. This method replaces the `torch.nn.Upsample()` operation with three operations supported by TensorRT 5.0.2.6 while keeping the preceding and subsequent steps unchanged. This improves model performance with minimal increase in computation and simplifies implementation. Deep learning models such as AlexNet, VGGNet, GoogleNet (pre-start network), and ResNet (residual network) can be used for deep learning scenarios such as image recognition, classification, and information prediction.

[0038] The implementation details of the technical solutions of the embodiments of this disclosure are described in detail below.

[0039] Figure 1 This illustration schematically depicts a flowchart of a deep learning model conversion deployment method according to an exemplary embodiment of this disclosure. Figure 1 As shown, the deep learning model conversion deployment method includes steps S101 to S104:

[0040] Step S101: Obtain the model file to be processed obtained by training the model using the first deep learning framework;

[0041] Step S102: Call the deconvolution function to perform an upsampling operation on the model file to be processed to obtain an initial model file;

[0042] Step S103: The initial model file is normalized using a normalization function to obtain an intermediate model file;

[0043] Step S104: Linearly correct the intermediate model file using an activation function to obtain the target model file, which is suitable for processing in the second deep learning framework.

[0044] In the technical solutions provided by some embodiments of this disclosure, upsampling, normalization, and linear correction are sequentially performed on the model file to be processed obtained by training a first deep learning framework to finally obtain a target model file. This target model file can then be applied to processing in a second deep learning framework. This allows the upsampling operation to be replaced with an operation applicable to the second deep learning framework without changing the input (the model file to be processed) and output (the target model file) of the existing upsampling operation, making it suitable for processing in the second deep learning framework. On the one hand, this avoids the development of customized plugins suitable for the second deep learning framework, resulting in less development and testing, simpler engineering implementation, and better model performance. On the other hand, it avoids the introduction of intermediate transition models, preventing loss of accuracy and performance, and eliminating the space occupied by intermediate model storage. Furthermore, it eliminates the need for version upgrades, thus avoiding the software and hardware incompatibility and upgrade testing costs associated with upgrades.

[0045] The following will describe in more detail each step of the deep learning model conversion and deployment method in this example embodiment, with reference to the accompanying drawings and embodiments.

[0046] In step S101, the model file to be processed is obtained by training the model using the first deep learning framework.

[0047] Furthermore, the first deep learning framework is a PyTorch model, and the model file to be processed is a pth file.

[0048] Specifically, PyTorch is an open-source deep learning framework that allows for the training, validation, and testing of deep learning models. PyTorch is used to train models and generate pth model files.

[0049] After obtaining the pth model file, the specific process of generating the TensorRT engine file using the pth model file is implemented through steps S102 to S104. That is, the functions implemented in steps S102 to S104 need to be able to replace the upsampling operation torch.nn.Upsample().

[0050] When using PyTorch in Python, you need to import torch to use PyTorch's functionality. Therefore, the upsampling operation is torch.nn.Upsample().

[0051] The `torch.nn.Upsample()` operation specifically performs an upsampling operation on an image. For example, an image of size axb will become 2a x 2b after the `torch.nn.Upsample(scale_factor=2)` operation. Similarly, after the `torch.nn.Upsample(scale_factor=4)` operation, the image size will become 4a x 4b. `torch.nn.Upsample()` is a common operation in deep learning models and is widely used, often in the intermediate or input layers of deep learning network structures.

[0052] When replacing the torch.nn.Upsample() operation, it is necessary to consider the input and output of the torch.nn.Upsample() operation to ensure that the input and output of the replacement are consistent with those of the torch.nn.Upsample() operation, and at the same time, to be as close as possible to the function performed by torch.nn.Upsample(), or even achieve better results.

[0053] Specifically, the input to `torch.nn.Upsample()` is a feature map of size `axb`. For a 1024x1024 image, `a` and `b` are both 1024. After the `torch.nn.Upsample(scale_factor=c)` operation, where `c` is an integer, if `c=2`, the output is a feature map of size `2a x 2b`. Generally, after the `torch.nn.Upsample(scale_factor=c)` upsampling operation, the output feature map size is `c*axc*b`.

[0054] In step S102, the deconvolution function is called to perform an upsampling operation on the model file to be processed to obtain an initial model file.

[0055] Furthermore, the deconvolution function is the ConvTranspose2d function. That is, the image upsampling operation is performed on the model file to be processed through deconvolution.

[0056] Figure 2 This schematically illustrates a flowchart of an upsampling operation method according to an exemplary embodiment of this disclosure. Figure 2As shown, the step of calling the deconvolution function to perform upsampling on the model file to be processed includes:

[0057] Step 201: Perform a fill operation on the feature map in the model file to be processed to obtain a new feature map;

[0058] Step 202: Randomly initialize convolutional kernels of the target size;

[0059] Step 203: Invoke the deconvolution function to perform convolution operation on the new feature map using the convolution kernel.

[0060] Specifically, in step 201, the input feature map is first padded to obtain a new feature map.

[0061] When s (stride) = 1, no interpolation operation is performed on the original feature map, only padding operation is performed, and padding_NewSize = (kernel_size - padding - 1). Here, padding is the padding value set during the convolution operation. If it is a valid convolution, padding = 0. kernel_size is the size of the convolution kernel in the transposed convolution.

[0062] When s>1, add s-1 zeros between every two pixels of the input, and then add (c+2*pk)mod(s) to the bottom and right sides of the input, where c is the size of the transposed convolution output calculated in the first step.

[0063] In step 202, a convolutional kernel of a certain size needs to be randomly initialized. It should be noted that the convolutional kernel is indeed randomly initialized, but it can be modified individually later, for example, by using a bilinear convolutional kernel. In this case, the parameters of the convolutional kernel are fixed and cannot be modified through learning.

[0064] In step 203, a convolution operation is performed. It's important to consider the size; the result should satisfy H_in = (H_out - 1) * sride - 2 * padding + kernel_size, where H_out is the size of the original feature map, and H_in is the size of the input image, which is the target size—the size we want to achieve through upsampling.

[0065] The above step S102 can be performed using the deconvolution function provided by PyTorch, namely `torch.nn.ConvTranspose2d`. The parameters of `torch.nn.ConvTranspose2d` are specifically set as follows: `kernel_size = c`, `stride = c`, `padding = 0`, `output_padding = 0`, and `bias = False`. `torch.nn.ConvTranspose2d` performs upsampling on the model file to be processed, achieving the same functionality as `torch.nn.Upsample()` while using a small number of variable internal parameters to achieve better upsampling results.

[0066] It should be noted that the parameters passed to the torch.nn.ConvTranspose2d module can change the value of the aforementioned kernel_size parameter, which will change the number of its internal parameters accordingly, achieving the same effect.

[0067] In step S103, the initial model file is normalized using a normalization function to obtain an intermediate model file.

[0068] Because the variable parameters inside the torch.nn.ConvTranspose2d operation can cause the calculated values ​​to differ from the original input data distribution, step S103 is required to eliminate cases where the mean is not 0 and the variance is not 1 in the output data, thereby maintaining consistency with the input data distribution.

[0069] Furthermore, the normalization function is the BatchNorm2d function. Adding BatchNorm2d after the deconvolution layer for data normalization prevents network performance instability due to excessively large data volumes before ReLU processing. The mathematical principle of the BatchNorm2d() function is shown in formula (1):

[0070]

[0071] Where γ is the adjusted variance and β is the adjusted mean. γ and β are the adjustments made to the pixels based on BN. The initial value of γ is 1 and the initial value of β is 0. ε is the stability coefficient, which is a value added to the denominator to improve the stability of the calculation. The default value is 1e-5 to avoid the denominator being 0. Var(x) is the variance of x and mean(x) is the mean of x.

[0072] Figure 3 This illustration schematically depicts a flowchart of a data normalization processing method according to an exemplary embodiment of this disclosure. For example... Figure 3As shown, the step of calling the normalization function to perform data normalization processing on the initial model file includes:

[0073] Step S301: Configure the internal parameters of the normalization function;

[0074] Step S302: Perform data normalization processing on the feature data in the initial model file based on the internal parameters.

[0075] In step S301, the parameters inside BatchNorm2d() are as follows:

[0076] 1. num_features: The typical input parameter is batch_size * num_features * height * width, which is the number of features.

[0077] 2.eps: A value added to the denominator for the purpose of ensuring calculation stability; the default value is 1e-5.

[0078] 3. momentum: An estimated parameter used to measure the mean and variance during operation;

[0079] 4. affine: When set to true, it will provide the learnable coefficient matrices gamma and beta.

[0080] Then, according to the configured internal parameters and the principle of formula (1), step S302 is executed to perform data normalization on the feature data in the initial model file, thereby obtaining the intermediate model file.

[0081] The above step S103 operation can be executed by calling the BatchNorm2d function provided by PyTorch, namely torch.nn.BatchNorm2d.

[0082] Step S104: Linearly correct the intermediate model file using an activation function to obtain the target model file, which is suitable for processing in the second deep learning framework.

[0083] Specifically, the second deep learning framework is a target version of the TensorRT model, and the target model file is a TensorRT engine file. In one embodiment of this disclosure, the target version of the TensorRT model may be TensorRT version 5.0.2.6.

[0084] TensorRT is currently a mainstream deep learning framework. Converting PyTorch models to TensorRT for deployment can accelerate computation. TensorRT version 5.0.2.6 supports TensorRT engine files.

[0085] After processing with torch.nn.BatchNorm2d, negative data values ​​may occur. For feature maps or images, negative pixel values ​​are meaningless. Therefore, step S104 is needed to perform a non-linear transformation on the data values ​​to further maintain consistency with the distribution of the input data, thereby improving the processing effect.

[0086] Figure 4 The illustration schematically shows a flowchart of a linear correction method according to an exemplary embodiment of this disclosure. Figure 4 As shown, the step of calling the activation function to perform linear correction on the intermediate model file includes:

[0087] Step S401: Obtain target feature data with negative data values ​​from the feature data in the intermediate model file;

[0088] Step S402: Modify the data values ​​of the target feature data based on the activation function.

[0089] Specifically, by calling the activation function to perform a nonlinear transformation on all values ​​in the intermediate model file, the target feature data with negative values ​​are reprocessed, and then one-sided suppression is performed.

[0090] The activation function can be the ReLU function. In deep neural networks, a rectified linear unit (ReLU) is typically used as the activation function for neurons. The ReLU function is actually a piecewise linear function; in step S402, negative values ​​are reassigned to 0, while positive values ​​remain unchanged.

[0091] Therefore, the above step S104 operation can be executed by calling the ReLU function provided by PyTorch, namely torch.nn.ReLU.

[0092] Alternatively, the activation function can also be the Sigmoid function. The Sigmoid function maps the output to the range [0,1], easily corresponding to probabilities and readily reflecting the probabilities of binary classification results. Another advantage is the relatively simple calculation of backpropagation. In step S402, negative values ​​are mapped to the positive real number domain.

[0093] Therefore, the above step S104 operation can be executed by calling the ReLU function provided by PyTorch, namely torch.nn.Sigmoid.

[0094] It should be noted that using other activation functions supported by TensorRT version 5.0.2.6 to perform linear correction on the data should also fall within the scope of protection of this disclosure.

[0095] Based on the above method, when converting a PyTorch model to the TensorRT engine, the three operations supported by TensorRT 5.0.2.6—torch.nn.ConvTranspose2d, torch.nn.BatchNorm2d, and torch.nn.ReLU (or torch.nn.Sigmoid, etc.)—are used to replace the torch.nn.Upsample() operation, thereby converting the PyTorch model into a model that can be mounted by TensorRT 5.0.2.6.

[0096] Compared to existing technologies for developing customized plugins for TensorRT 5.0.2.6, this approach requires less engineering development and testing, while delivering superior model performance. Compared to technologies that generate intermediate ONNX models from PTH and then convert them into TensorRTengine files, this approach avoids the loss of accuracy and performance caused by introducing intermediate models, eliminates the space occupied by intermediate storage, and avoids the computation required to generate intermediate models. Compared to upgrading TensorRT to version 7.0 or higher, this approach eliminates the hardware and software incompatibility and upgrade testing costs associated with TensorRT version upgrades.

[0097] Figure 5 This schematic diagram illustrates the composition of a deep learning model conversion deployment apparatus according to an exemplary embodiment of the present disclosure, such as... Figure 5 As shown, the deep learning model conversion and deployment device 500 may include an acquisition module 501, a deconvolution module 502, a normalization module 503, and an activation function module 504. Wherein:

[0098] Module 501 is used to acquire the model file to be processed obtained by training the model using the first deep learning framework.

[0099] Deconvolution module 502 is used to call the deconvolution function to perform an upsampling operation on the model file to be processed to obtain an initial model file;

[0100] The normalization module 503 is used to perform data normalization processing on the initial model file using a normalization function to obtain an intermediate model file;

[0101] The activation function module 504 is used to call the activation function to linearly correct the intermediate model file to obtain the target model file, so that it can be processed in the second deep learning framework.

[0102] According to an exemplary embodiment of this disclosure, the first deep learning framework is a PyTorch model, and the model file to be processed is a pth file.

[0103] According to an exemplary embodiment of this disclosure, the second deep learning framework is a TensorRT model of version 5.0.2.6, and the target model file is a TensorRT engine file.

[0104] According to an exemplary embodiment of this disclosure, the deconvolution function is a ConvTranspose2d function, and the deconvolution module 502 is used to fill the feature map in the model file to be processed to obtain a new feature map; randomly initialize the convolution kernel of the target size; and call the deconvolution function to perform convolution operation on the new feature map using the convolution kernel.

[0105] According to an exemplary embodiment of this disclosure, the normalization function is a BatchNorm2d function, and the normalization module 503 is used to configure the internal parameters of the normalization function; and to perform data normalization processing on the feature data in the initial model file based on the internal parameters.

[0106] According to an exemplary embodiment of this disclosure, the activation function module 504 is used to obtain target feature data with negative data values ​​from the feature data in the intermediate model file; and to modify the data values ​​of the target feature data based on the activation function.

[0107] According to an exemplary embodiment of this disclosure, the activation function is a ReLU function or a Sigmoid function.

[0108] The specific details of each module in the aforementioned deep learning model conversion and deployment device 500 have been described in detail in the corresponding deep learning model conversion and deployment methods, so they will not be repeated here.

[0109] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0110] In an exemplary embodiment of this disclosure, a storage medium capable of implementing the above-described method is also provided. Figure 6 This schematic diagram illustrates a computer-readable storage medium according to an exemplary embodiment of the present disclosure, such as... Figure 6 As shown, a program product 600 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a mobile phone. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0111] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided. Figure 7 The schematic diagram illustrates the structure of a computer system of an electronic device according to an exemplary embodiment of the present disclosure.

[0112] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0113] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage section 708 into Random Access Memory (RAM) 703. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0114] The following components are connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 710 as needed so that computer programs read from it can be installed into storage section 708 as needed.

[0115] In particular, according to embodiments of this disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs various functions defined in the system of this disclosure.

[0116] It should be noted that the computer-readable medium shown in the embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0119] In another aspect, this disclosure also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0120] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0121] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0122] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0123] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for deploying and converting deep learning models, characterized in that, include: Obtain the model file to be processed obtained by training the model using a first deep learning framework; the first deep learning framework is a PyTorch model. The model file to be processed is a pth file; The deconvolution function is called to perform an upsampling operation on the model file to be processed, resulting in an initial model file; The initial model file is normalized using a normalization function to obtain an intermediate model file; The intermediate model file is linearly corrected using an activation function to obtain the target model file, which is suitable for processing in a second deep learning framework; the second deep learning framework is the target version of the TensorRT model; the target model file is the TensorRT engine file.

2. The deep learning model conversion and deployment method according to claim 1, characterized in that, The deconvolution function is the ConvTranspose2d function, and the upsampling operation on the model file to be processed by calling the deconvolution function includes: The feature map in the model file to be processed is filled to obtain a new feature map; Randomly initialize convolutional kernels of the target size; The deconvolution function is invoked to perform a convolution operation on the new feature map using the convolution kernel.

3. The deep learning model conversion and deployment method according to claim 1, characterized in that, The normalization function is the BatchNorm2d function. The process of normalizing the initial model file using the normalization function includes: Configure the internal parameters of the normalization function; The feature data in the initial model file is normalized based on the internal parameters.

4. The deep learning model conversion and deployment method according to claim 1, characterized in that, The linear correction of the intermediate model file using an activation function includes: Obtain the target feature data with negative data values ​​from the intermediate model file; The data values ​​of the target feature data are changed based on the activation function.

5. The deep learning model conversion and deployment method according to claim 1, characterized in that, The activation function is either the ReLU function or the Sigmoid function.

6. A deep learning model conversion and deployment device, characterized in that, include: The acquisition module is used to acquire the model file to be processed obtained by training the model using a first deep learning framework; the first deep learning framework is a PyTorch model. The model file to be processed is a pth file; The deconvolution module is used to call the deconvolution function to perform an upsampling operation on the model file to be processed, so as to obtain an initial model file; The normalization module is used to perform data normalization processing on the initial model file using a normalization function to obtain an intermediate model file; The activation function module is used to call the activation function to linearly correct the intermediate model file to obtain the target model file, which is suitable for processing in the second deep learning framework; the second deep learning framework is the target version of the TensorRT model; the target model file is the TensorRT engine file.

7. A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the deep learning model conversion deployment method as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the deep learning model conversion deployment method as described in any one of claims 1 to 5.

Images