Model conversion optimization device, method and readable storage medium for deep learning model

Abstract

A model conversion optimization device, method and computer-readable storage medium for a deep learning model, the method comprising: performing model conversion on a first deep learning model to obtain a second deep learning model; obtaining weight arrangement methods of the two models according to the deep learning frameworks of the first and second deep learning models respectively; performing model quantization on the first and second deep learning models; obtaining weight similarity of the two models based on the weight arrangement methods of the first and second deep learning models and the model quantization results to construct a weight analysis report; testing the first and second deep learning models according to a preset test set to construct a model performance estimation report; obtaining optimization suggestions for the second deep learning model according to the weight and model performance report; and processing the data to be processed using the optimized second deep learning model. The present invention can achieve in-depth analysis before and after model conversion to avoid poor performance after model conversion.

Description

Technical Field

[0001] The present invention relates to the field of model conversion optimization technology, and in particular to a model conversion optimization device, method and computer-readable storage medium for a deep learning model. Background Art

[0002] Existing deep learning frameworks include TensorFlow, Caffe, NCNN, Pytorch and other frameworks. Different deep learning frameworks are often used in the process from algorithm development to algorithm deployment. For example, the Tensorflow framework is used for algorithm development and debugging, while the NCNN framework may be used in the terminal deployment stage, which leads to the problem of model conversion. In order to ensure that the model can still maintain good prediction accuracy after conversion, it is necessary to analyze the model before and after conversion. However, the existing model analysis is generally performed by personnel with rich deep learning knowledge, which is overly dependent on the professionalism of analysts, and has the problems of high time consumption and low efficiency, resulting in poor performance of the model finally deployed on the terminal. Summary of the invention

[0003] In view of this, it is necessary to provide a model conversion optimization device, method and computer-readable storage medium for a deep learning model, which can automatically perform in-depth analysis before and after the model conversion, and avoid the situation where the performance of the converted model is poor to the greatest extent.

[0004] An embodiment of the present invention provides a model conversion optimization method for a deep learning model, the method comprising:

[0005] Performing model conversion on the first deep learning model to obtain a second deep learning model, wherein the first deep learning model and the second deep learning model have different deep learning frameworks;

[0006] Obtaining a weight arrangement manner of the first deep learning model according to the deep learning framework of the first deep learning model and obtaining a weight arrangement manner of the second deep learning model according to the deep learning framework of the second deep learning model;

[0007] Performing model quantization on the first deep learning model and the second deep learning model;

[0008] Based on the weight arrangement method and model quantization result of the first deep learning model, and the weight arrangement method and model quantization result of the second deep learning model, the weight similarity between the first deep learning model and the second deep learning model is analyzed to construct a weight analysis report;

[0009] Testing the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report;

[0010] Obtaining optimization suggestions for the second deep learning model according to the weight analysis report and the model performance estimation report to optimize the second deep learning model; and

[0011] The optimized second deep learning model is used to process the data to be processed.

[0012] Preferably, the step of quantizing the first deep learning model and the second deep learning model comprises:

[0013] Obtaining weight distribution information of the first deep learning model based on the weight arrangement method of the first deep learning model, and performing model quantization on the first deep learning model according to the weight distribution information of the first deep learning model; and

[0014] Based on the weight arrangement method of the second deep learning model, weight distribution information of the second deep learning model is obtained, and model quantization is performed on the second deep learning model according to the weight distribution information of the second deep learning model.

[0015] Preferably, the weight analysis report also includes weight distribution information of the first deep learning model and weight distribution information of the second deep learning model.

[0016] Preferably, the step of performing model quantization on the first deep learning model according to the weight distribution information of the first deep learning model includes:

[0017] The number of bits and precision for quantizing the first deep learning model are determined according to the weight distribution information of the first deep learning model, so as to quantize the first deep learning model based on the determined number of bits and precision.

[0018] Preferably, the step of performing model quantization on the second deep learning model according to the weight distribution information of the second deep learning model includes:

[0019] The number of bits and precision for quantizing the second deep learning model are determined according to the weight distribution information of the second deep learning model, so as to quantize the second deep learning model based on the determined number of bits and precision.

[0020] Preferably, the data of the preset test set includes a training sample library for performing model training on the first deep learning model and the second deep learning model, and the samples of the training sample library are divided into a training set and a validation set.

[0021] Preferably, the data of the preset test set also includes a field sample set not included in the training sample library and / or an amplified sample set generated by amplifying the verification set.

[0022] Preferably, the step of testing the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report includes:

[0023] Testing the first deep learning model and the second deep learning model using the training sample library, the field sample set, and the augmented sample set; and

[0024] The model performance estimation report is constructed based on the first test result obtained from the training sample library, the second test result obtained from the field sample set, and the third test result obtained from the augmented sample set.

[0025] One embodiment of the present invention provides a model conversion optimization device for a deep learning model, the device comprising a processor and a memory, the memory storing a plurality of computer programs, and the processor being used to implement the steps of the model conversion optimization method for the above-mentioned deep learning model when executing the computer programs stored in the memory.

[0026] One embodiment of the present invention also provides a computer-readable storage medium, which stores multiple instructions, and the multiple instructions can be executed by one or more processors to implement the steps of the above-mentioned model conversion optimization method of the deep learning model.

[0027] Compared with the existing technology, the model conversion optimization device, method and computer-readable storage medium of the above-mentioned deep learning model can automatically perform in-depth analysis of the deep learning model before and after conversion, with high analysis efficiency, and can give model optimization suggestions based on the analysis results, so as to avoid the situation of poor performance of the converted model to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a functional module diagram of a model conversion optimization device according to one embodiment of the present invention.

[0029] Figure 2 It is a functional module diagram of a model conversion optimization program according to one embodiment of the present invention.

[0030] Figure 3 It is a flow chart of a model conversion optimization method according to one embodiment of the present invention.

[0031] Main component symbols

[0032] Memory 10 processor 20 Model Conversion Optimizer 30 Conversion Module 101 Determine the module 102 Quantization module 103 First building block 104 Second building block 105 Optimization Module 106 Processing Module 107 Model conversion optimization device 100

[0033] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] It should be further noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0036] See also Figure 1 , which is a schematic diagram of a preferred embodiment of the model conversion optimization device of the present invention.

[0037] The model conversion optimization device 100 can provide model optimization suggestions for the model conversion of deep learning models between different frameworks, and can verify the prediction effect of the model before and after the conversion. The model conversion optimization device 100 may include a memory 10, a processor 20, and a model conversion optimization program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the model conversion optimization program 30, the steps in the model conversion optimization method embodiment are implemented, such as Figure 3 Alternatively, the processor 20 executes the model conversion optimization program 30 to implement Figure 2 The functions of each module in the module, such as modules 101 to 107.

[0038] The model conversion optimization program 30 may be divided into one or more modules, which are stored in the memory 10 and executed by the processor 20 to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the model conversion optimization program 30 in the model conversion optimization device 100. For example, the model conversion optimization program 30 may be divided into Figure 2The conversion module 101, the determination module 102, the quantization module 103, the first construction module 104, the second construction module 105, the optimization module 106 and the processing module 107 in the embodiment of the present invention are shown in the following table. Figure 2 The functions of each module in.

[0039] Those skilled in the art will understand that the schematic diagram is merely an example of the model conversion optimization device 100 and does not constitute a limitation of the model conversion optimization device 100. The device may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the model conversion optimization device 100 may also include input and output devices, communication modules, buses, etc.

[0040] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor 20 may also be any conventional processor, etc. The processor 20 may use various interfaces and buses to connect various parts of the model conversion optimization device 100.

[0041] The memory 10 can be used to store the model conversion optimization program 30 and / or modules. The processor 20 implements various functions of the model conversion optimization device 100 by running or executing the computer program and / or module stored in the memory 10, and calling the data stored in the memory 10. The memory 10 may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0042] Figure 2 It is a functional module diagram of a preferred embodiment of the model conversion optimization program of the present invention.

[0043] See also Figure 2As shown, the model conversion optimization program 30 may include a conversion module 101, a determination module 102, a quantification module 103, a first construction module 104, a second construction module 105, an optimization module 106, and a processing module 107. In one embodiment, the above modules may be programmable software instructions stored in the memory 10 and can be called and executed by the processor 20. It is understood that in other embodiments, the above modules may also be program instructions or firmware solidified in the processor 20.

[0044] The conversion module 101 is used to perform model conversion on the first deep learning model to obtain a second deep learning model.

[0045] In one embodiment, the first deep learning model and the second deep learning model preferably have different deep learning frameworks. For example, in the model development stage, the first deep learning model is developed and tested using the Tensorflow framework, and in the terminal deployment stage, the model of the NCNN framework needs to be deployed to the terminal, so it is necessary to convert the first deep learning model with the Tensorflow framework into a second deep learning model with the NCNN framework, and then realize the deployment of the second deep learning model to the terminal. The terminal can be an electronic device such as a mobile phone or a tablet computer. The conversion module 101 can perform model conversion on the first deep learning model based on a preset deep learning model conversion tool to obtain a second deep learning model. The preset deep learning model conversion tool can be an existing model converter, which is not limited here.

[0046] It can be understood that the first deep learning model and the second deep learning model have the same model function, and the model functions of the first deep learning model and the second deep learning model can be determined according to actual development requirements. For example, the first deep learning model and the second deep learning model can be used for image recognition, speech recognition, image processing, data mining, or natural language processing.

[0047] It can be understood that both the first deep learning model and the second deep learning model can use samples in the training sample library for model training, and the samples in the training sample library are divided into a training set and a validation set. The training set is used for model training, and the validation set is used to test the performance of the model.

[0048] The determination module 102 is used to obtain a weight arrangement method of the first deep learning model according to a deep learning framework of the first deep learning model and to obtain a weight arrangement method of the second deep learning model according to a deep learning framework of the second deep learning model.

[0049] In one embodiment, when the model conversion of the first deep learning model is completed to obtain the second deep learning model, the determination module 102 can obtain the weight arrangement of the second deep learning model according to the deep learning framework of the second deep learning model. The determination module 102 can also obtain the weight arrangement of the first deep learning model according to the deep learning framework of the first deep learning model. The weight arrangement of the first deep learning model can represent the arrangement of weight parameters in the first deep learning model, and the weight arrangement of the second deep learning model can represent the arrangement of weight parameters in the second deep learning model.

[0050] The quantization module 103 is used to perform model quantization on the first deep learning model and the second deep learning model.

[0051] In one embodiment, the quantization module 103 may perform a quantization operation of a specified quantization type on the first deep learning model and the second deep learning model. For example, the quantization type may be to quantize only the model weights, or to quantize both the model weights and the model activation outputs.

[0052] In one embodiment, the quantization module 103 can obtain the weight distribution information of the first deep learning model based on the weight arrangement method of the first deep learning model, and then perform model quantization on the first deep learning model according to the weight distribution information of the first deep learning model. The weight distribution information of the first deep learning model can characterize the distribution of weight parameters in the first deep learning model. The quantization module 103 can also obtain the weight distribution information of the second deep learning model based on the weight arrangement method of the second deep learning model, and then perform model quantization on the second deep learning model according to the weight distribution information of the second deep learning model. The weight distribution information of the second deep learning model can characterize the distribution of weight parameters in the second deep learning model.

[0053] In one embodiment, the specific implementation method of the quantization module 103 performing model quantization on the first deep learning model according to the weight distribution information of the first deep learning model may be: the quantization module 103 may determine the number of bits (bits) and precision (such as int or float) for model quantization of the first deep learning model according to the weight distribution information of the first deep learning model, and then perform model quantization on the first deep learning model based on the determined number of bits and precision. The specific implementation method of the quantization module 103 performing model quantization on the second deep learning model according to the weight distribution information of the second deep learning model may be: the quantization module 103 determines the number of bits and precision for model quantization of the second deep learning model according to the weight distribution information of the second deep learning model, and then performs model quantization on the second deep learning model based on the determined number of bits and precision.

[0054] The first construction module 104 is used to obtain the weight similarity between the first deep learning model and the second deep learning model based on the weight arrangement method and model quantization results of the first deep learning model, and the weight arrangement method and model quantization results of the second deep learning model, so as to construct a weight analysis report.

[0055] In one embodiment, when the quantization module 103 completes the model quantization of the first deep learning model and the second deep learning model, the first construction module 104 can obtain the weight similarity between the first deep learning model and the second deep learning model based on the weight arrangement method and model quantization result of the first deep learning model, and the weight arrangement method and model quantization result of the second deep learning model, and then construct the weight analysis report based on the weight similarity obtained by analysis.

[0056] In one embodiment, the weight analysis report may further include weight distribution information of the first deep learning model and weight distribution information of the second deep learning model.

[0057] The second construction module 105 tests the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report.

[0058] In one embodiment, the model performance estimation report includes the test results of the model prediction performance of the first deep learning model and the test results of the model prediction performance of the second deep learning model. The data of the preset test set may include a training sample library for model training of the first deep learning model and the second deep learning model, and the samples of the training sample library are divided into a training set and a validation set.

[0059] For example, the functions of the first deep learning model and the second deep learning model are image recognition, and the training sample library includes 1,000 image samples, of which the training set includes 800 image samples and the verification set includes 200 image samples. During the model training process of the first deep learning model, the first deep learning model is trained using the image samples of the training set, and the prediction effect of the trained first deep learning model is verified using the image samples of the verification set. During the model training process of the second deep learning model, the second deep learning model is also trained using the image samples of the training set, and the prediction effect of the trained second deep learning model is verified using the image samples of the verification set. The preset test set includes all image samples of the training sample library, that is, the second construction module 105 uses 1,000 image samples to test the first deep learning model and uses 1,000 image samples to test the second deep learning model, so as to construct the model performance estimation report according to the test results.

[0060] In one embodiment, the data of the preset test set may also include a field sample set that is not included in the training sample library, that is, sample data that the model has not been exposed to during the training process. When the second deep learning model is deployed on the terminal, the field sample set may refer to sample data obtained by the terminal in an actual test use scenario. For example, the second deep learning model is a face recognition model used for face recognition, and the field sample set may be a face image collected by the terminal during use.

[0061] In one embodiment, the data of the preset test set may also include an amplified sample set generated by amplifying the verification set. For example, the samples in the verification set may be flipped, mirrored, etc., to obtain the amplified sample set.

[0062] For example, the verification set may be 200 image samples, and 2000 image samples are obtained by performing operations such as translation, flipping, rotation, adjusting contrast, Gaussian noise, and color transformation on multiple image samples in the verification set. The 2000 image samples constitute the augmented sample set.

[0063] In one embodiment, the second construction module 105 tests the first deep learning model and the second deep learning model respectively according to a preset test set, and the specific method for constructing a model performance estimation report may include: the second construction module 105 uses the training sample library, the field sample set and the augmented sample set to test the first deep learning model and the second deep learning model, and then constructs the model performance estimation report according to the first test result obtained from the training sample library, the second test result obtained from the field sample set, and the third test result obtained from the augmented sample set.

[0064] The optimization module 106 is used to obtain optimization suggestions for the second deep learning model based on the weight analysis report and the model performance estimation report to optimize the second deep learning model.

[0065] In one embodiment, when the weight analysis report and the model performance estimation report are obtained, the optimization module 106 can obtain optimization suggestions for the second deep learning model based on the weight analysis report and the model performance estimation report to optimize the second deep learning model, thereby improving the model prediction accuracy of the second deep learning model deployed on the terminal and saving the computing resources occupied by the model.

[0066] In one embodiment, when the optimization module 106 obtains optimization suggestions for the second deep learning model based on the weight analysis report and the model performance estimation report, the model developer or the end user can control the second deep learning model to execute the optimization suggestions.

[0067] The processing module 107 is used to process the data to be processed using the optimized second deep learning model.

[0068] In one embodiment, the data to be processed may refer to input data acquired by a terminal in which the second deep learning model is deployed in an actual usage scenario and input into the second deep learning model. After the second deep learning model executes the optimization suggestion, the processing module 107 may process the data to be processed using the optimized second deep learning model to obtain a processing result corresponding to the data to be processed.

[0069] Figure 3 The flowchart of the model conversion optimization method of the deep learning model in one embodiment of the present invention is shown in FIG. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0070] Step S300: performing model conversion on the first deep learning model to obtain a second deep learning model.

[0071] In one embodiment, the first deep learning model and the second deep learning model preferably have different deep learning frameworks. For example, in the model development stage, the first deep learning model is developed and tested using the Tensorflow framework, and in the terminal deployment stage, the model of the NCNN framework needs to be deployed to the terminal, so it is necessary to convert the first deep learning model with the Tensorflow framework into a second deep learning model with the NCNN framework, and then realize the deployment of the second deep learning model to the terminal. The terminal can be an electronic device such as a mobile phone or a tablet computer. The first deep learning model can be converted based on a preset deep learning model conversion tool to obtain a second deep learning model. The preset deep learning model conversion tool can be an existing model converter, which is not limited here.

[0072] It can be understood that the first deep learning model and the second deep learning model have the same model function, and the model functions of the first deep learning model and the second deep learning model can be determined according to actual development requirements. For example, the first deep learning model and the second deep learning model can be used for image recognition, speech recognition, image processing, data mining, or natural language processing.

[0073] Step S302: obtaining a weight arrangement method of the first deep learning model according to the deep learning framework of the first deep learning model and obtaining a weight arrangement method of the second deep learning model according to the deep learning framework of the second deep learning model.

[0074] In one embodiment, when the model conversion of the first deep learning model is completed to obtain the second deep learning model, the weight arrangement of the second deep learning model can be obtained according to the deep learning framework of the second deep learning model. The weight arrangement of the first deep learning model can also be obtained according to the deep learning framework of the first deep learning model. The weight arrangement of the first deep learning model can represent the arrangement of weight parameters in the first deep learning model, and the weight arrangement of the second deep learning model can represent the arrangement of weight parameters in the second deep learning model.

[0075] Step S304: perform model quantization on the first deep learning model and the second deep learning model.

[0076] In one embodiment, a quantization operation of a specified quantization type may be performed on the first deep learning model and the second deep learning model. For example, the quantization type may be to quantize only the model weights, or to quantize both the model weights and the model activation outputs.

[0077] In one embodiment, the weight distribution information of the first deep learning model can be obtained based on the weight arrangement method of the first deep learning model, and then the first deep learning model can be model quantized according to the weight distribution information of the first deep learning model. The weight distribution information of the first deep learning model can characterize the distribution of weight parameters in the first deep learning model. The weight distribution information of the second deep learning model can also be obtained based on the weight arrangement method of the second deep learning model, and then the second deep learning model can be model quantized according to the weight distribution information of the second deep learning model. The weight distribution information of the second deep learning model can characterize the distribution of weight parameters in the second deep learning model.

[0078] In one embodiment, the step of model quantizing the first deep learning model according to the weight distribution information of the first deep learning model may include: determining the number of bits and precision (such as int or float) for model quantizing the first deep learning model according to the weight distribution information of the first deep learning model, and then model quantizing the first deep learning model based on the determined number of bits and precision. The step of model quantizing the second deep learning model according to the weight distribution information of the second deep learning model may include: determining the number of bits and precision for model quantizing the second deep learning model according to the weight distribution information of the second deep learning model, and then model quantizing the second deep learning model based on the determined number of bits and precision.

[0079] Step S306, based on the weight arrangement method and model quantization results of the first deep learning model, and the weight arrangement method and model quantization results of the second deep learning model, the weight similarity between the first deep learning model and the second deep learning model is analyzed to construct a weight analysis report.

[0080] In one embodiment, when the model quantization of the first deep learning model and the second deep learning model is completed, the weight similarity between the first deep learning model and the second deep learning model can be analyzed based on the weight arrangement method and model quantization result of the first deep learning model, and the weight arrangement method and model quantization result of the second deep learning model, and then the weight analysis report can be constructed based on the weight similarity obtained by analysis.

[0081] In one embodiment, the weight analysis report may further include weight distribution information of the first deep learning model and weight distribution information of the second deep learning model.

[0082] Step S308: Test the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report.

[0083] In one embodiment, the model performance estimation report includes the test results of the model prediction performance of the first deep learning model and the test results of the model prediction performance of the second deep learning model. The data of the preset test set may include a training sample library for model training of the first deep learning model and the second deep learning model, and the samples of the training sample library are divided into a training set and a validation set.

[0084] For example, the functions of the first deep learning model and the second deep learning model are image recognition, and the training sample library includes 1,000 image samples, of which the training set includes 800 image samples and the verification set includes 200 image samples. During the model training process of the first deep learning model, the first deep learning model is trained using the image samples of the training set, and the prediction effect of the trained first deep learning model is verified using the image samples of the verification set. During the model training process of the second deep learning model, the second deep learning model is also trained using the image samples of the training set, and the prediction effect of the trained second deep learning model is verified using the image samples of the verification set. The preset test set includes all image samples in the training sample library, that is, the first deep learning model is tested using 1,000 image samples and the second deep learning model is tested using 1,000 image samples, so as to construct the model performance estimation report based on the test results.

[0085] In one embodiment, the data of the preset test set may also include a field sample set that is not included in the training sample library, that is, sample data that the model has not been exposed to during the training process. When the second deep learning model is deployed on the terminal, the field sample set may refer to sample data obtained by the terminal in an actual test use scenario. For example, the second deep learning model is a face recognition model used for face recognition, and the field sample set may be a face image collected by the terminal during use.

[0086] In one embodiment, the data of the preset test set may also include an amplified sample set generated by amplifying the verification set. For example, the samples in the verification set may be flipped, mirrored, etc., to obtain the amplified sample set.

[0087] For example, the verification set may be 200 image samples, and 2000 image samples are obtained by performing operations such as translation, flipping, rotation, adjusting contrast, Gaussian noise, and color transformation on multiple image samples in the verification set. The 2000 image samples constitute the augmented sample set.

[0088] In one embodiment, the step of testing the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report may include: using the training sample library, the field sample set and the augmented sample set to test the first deep learning model and the second deep learning model, and then constructing the model performance estimation report based on the first test result obtained from the training sample library, the second test result obtained from the field sample set, and the third test result obtained from the augmented sample set.

[0089] Step S310: Obtain optimization suggestions for the second deep learning model based on the weight analysis report and the model performance estimation report to optimize the second deep learning model.

[0090] In one embodiment, when the weight analysis report and the model performance estimation report are obtained, optimization suggestions for the second deep learning model can be obtained based on the weight analysis report and the model performance estimation report to optimize the second deep learning model, thereby improving the model prediction accuracy of the second deep learning model deployed on the terminal and saving the computing resources occupied by the model.

[0091] In one embodiment, when optimization suggestions for the second deep learning model are obtained based on the weight analysis report and the model performance estimation report, the model developer or the end user can control the second deep learning model to execute the optimization suggestions.

[0092] Step S312: Process the data to be processed using the optimized second deep learning model.

[0093] In one embodiment, the data to be processed may refer to input data acquired by a terminal in which the second deep learning model is deployed in an actual usage scenario and input into the second deep learning model. After the second deep learning model executes the optimization suggestion, the data to be processed may be processed using the optimized second deep learning model to obtain a processing result corresponding to the data to be processed.

[0094] The model conversion optimization device, method and computer-readable storage medium of the above-mentioned deep learning model can automatically perform in-depth analysis of the deep learning model before and after conversion, with high analysis efficiency, and can give model optimization suggestions based on the analysis results, so as to minimize the occurrence of poor performance of the converted model.

[0095] For those skilled in the art, other corresponding changes or adjustments can be made based on the inventive scheme and inventive concept of the present invention in combination with actual production needs, and these changes and adjustments should all fall within the scope disclosed by the present invention.

Claims

1. A model conversion optimization method for a deep learning model, characterized in that: The method comprises: Performing model conversion on the first deep learning model to obtain a second deep learning model, wherein the first deep learning model and the second deep learning model have different deep learning frameworks; Obtaining a weight arrangement manner of the first deep learning model according to the deep learning framework of the first deep learning model and obtaining a weight arrangement manner of the second deep learning model according to the deep learning framework of the second deep learning model; Performing model quantization on the first deep learning model and the second deep learning model; Based on the weight arrangement method and model quantization result of the first deep learning model, and the weight arrangement method and model quantization result of the second deep learning model, the weight similarity between the first deep learning model and the second deep learning model is analyzed to construct a weight analysis report; Testing the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report; Obtaining optimization suggestions for the second deep learning model according to the weight analysis report and the model performance estimation report to optimize the second deep learning model; and The optimized second deep learning model is used to process the data to be processed, where the data to be processed is image data or speech data.

2. The method according to claim 1, characterized in that The step of quantizing the first deep learning model and the second deep learning model comprises: Obtaining weight distribution information of the first deep learning model based on the weight arrangement method of the first deep learning model, and performing model quantization on the first deep learning model according to the weight distribution information of the first deep learning model; and Based on the weight arrangement method of the second deep learning model, weight distribution information of the second deep learning model is obtained, and model quantization is performed on the second deep learning model according to the weight distribution information of the second deep learning model.

3. The method according to claim 2, characterized in that The weight analysis report also includes weight distribution information of the first deep learning model and weight distribution information of the second deep learning model.

4. The method according to claim 2, characterized in that The step of performing model quantization on the first deep learning model according to the weight distribution information of the first deep learning model includes: The number of bits and precision for quantizing the first deep learning model are determined according to the weight distribution information of the first deep learning model, so as to quantize the first deep learning model based on the determined number of bits and precision.

5. The method according to claim 2, characterized in that The step of performing model quantization on the second deep learning model according to the weight distribution information of the second deep learning model includes: The number of bits and precision for quantizing the second deep learning model are determined according to the weight distribution information of the second deep learning model, so as to quantize the second deep learning model based on the determined number of bits and precision.

6. The method according to claim 1, characterized in that The data of the preset test set includes a training sample library used to perform model training on the first deep learning model and the second deep learning model, and the samples of the training sample library are divided into a training set and a validation set.

7. The method according to claim 6, characterized in that The data of the preset test set also includes a field sample set not included in the training sample library and / or an amplified sample set generated by amplifying the verification set.

8. The method according to claim 7, characterized in that The step of testing the first deep learning model and the second deep learning model respectively according to a preset test set to construct a model performance estimation report includes: Testing the first deep learning model and the second deep learning model using the training sample library, the field sample set, and the augmented sample set; and The model performance estimation report is constructed based on the first test result obtained from the training sample library, the second test result obtained from the field sample set, and the third test result obtained from the augmented sample set.

9. A model conversion optimization device for a deep learning model, the device comprising a processor and a memory, wherein the memory stores a plurality of computer programs, characterized in that: The processor is used to implement the steps of the model conversion optimization method of the deep learning model as described in any one of claims 1 to 8 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores multiple instructions, and the multiple instructions can be executed by one or more processors to implement the steps of the model conversion optimization method of the deep learning model as described in any one of claims 1 to 8.

Images