Method and apparatus for optimizing deep neural network model, electronic device and medium

Abstract

The application provides an optimization method and device of a deep neural network model, electronic equipment and a medium. The optimization method separately extracts each convolution layer of the deep neural network model, and separately outputs the extracted convolution layer on a target platform. The number of convolution kernels of each convolution layer is changed through traversal, and the time delay data of the convolution layer with different numbers of convolution kernels on the target platform is collected. The overall time delay of the deep neural network model with multiple different structures is calculated through a simulator, and the deep neural network model with an overall time delay meeting a target time delay condition is determined. The selected deep neural network model is retrained to obtain a target deep neural network model special for the target platform. The time delay data collected by the optimization method can perceive platform attributes, provide customized optimization basis for a specific platform, and obtain a deep neural network model special for the target platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a deep neural network model optimization method and device, electronic equipment and medium. BACKGROUND

[0002] Deep learning based on deep neural networks (DNN) has achieved breakthrough results in many real-world applications. At the same time, the high complexity of deep neural network algorithm models requires very demanding computing and storage resources. Especially on resource-constrained embedded devices, even if a general deep neural network model can be deployed, it is difficult to meet the speed delay requirements of actual applications.

[0003] To address the above problems, many works have tried to propose solutions from both software algorithms and hardware platforms. For example, various deep neural network optimization techniques and various new hardware accelerator platforms dedicated to deep neural network fields. However, few works can combine the two organically. Current advanced neural network optimization techniques are mostly algorithm model level optimizations. Since the hardware architecture features of different platforms are very different, these optimization techniques cannot "perceive" these feature information, ultimately leading to their inability to achieve the expected optimization effect on all platforms. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a deep neural network model optimization method, device, electronic equipment and medium, which can perceive the attributes of the target platform, simulate the latency of a certain optimized deep neural network model, provide customized optimization basis for the target platform, and generate a deep neural network model that is unique to the target platform.

[0005] The deep neural network model optimization method provided by the present application comprises:

[0006] Each convolutional layer of the deep neural network model is extracted and output separately on the target platform;

[0007] For each convolutional layer, the number of convolutional kernels of the convolutional layer is changed, the latency data of the convolutional layer with different numbers of convolutional kernels on the target platform is collected, and a latency data set including the latency data of each convolutional layer with different numbers of convolutional kernels is obtained;

[0008] Based on the latency data set, the overall latency of deep neural network models with multiple different structures is calculated by the simulator, and a deep neural network model whose overall latency meets the target latency condition is determined, wherein the number of convolutional kernels in at least one convolutional layer of the deep neural network model with different structures is different;

[0009] retraining the screened deep neural network model to obtain a target deep neural network model specific to the target platform.

[0010] In some embodiments, the optimization method of the deep neural network model includes: based on the latency dataset, calculating overall latencies of deep neural network models with different structures respectively by the simulator, and determining a deep neural network model with an overall latency satisfying a target latency condition.

[0011] determining a total target speedup ratio according to a target accuracy requirement of the deep neural network model; the total target speedup ratio represents a performance improvement degree of the deep neural network model before and after optimization;

[0012] performing multi-round iterative optimization on the deep neural network model by the simulator, and determining a sub-target speedup ratio of each iteration round according to the iteration round and the total target speedup ratio;

[0013] In the current iteration round, the simulator calculates an overall latency of a deep neural network model with at least one structure based on the latency dataset, and determines a deep neural network model satisfying the sub-target speedup ratio of the current round and a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall latency.

[0014] performing iterative optimization on the deep neural network model after the current round of optimization in the next round until the overall latency of the deep neural network model satisfies the total target speedup ratio.

[0015] In some embodiments, the optimization method of the deep neural network model includes: in the current iteration round, the simulator calculates overall latencies of deep neural network models with different structures respectively based on the latency dataset, and determines a deep neural network model satisfying the sub-target speedup ratio of the current round and a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall latencies.

[0016] iterating through k trimmable convolutional layers in the deep neural network model by the simulator, and for each convolutional layer, determining a number of convolutional kernels to be reserved in the convolutional layer to satisfy the sub-target speedup ratio of the current iteration round;

[0017] determining the convolutional kernels to be reserved based on an influence degree of different convolutional kernels in the convolutional layer on the accuracy of the deep neural network model, to obtain a candidate deep neural network model corresponding to the convolutional layer and satisfying the preset accuracy condition;

[0018] from k candidate deep neural network models corresponding to the k trimmable convolutional layers, screening out a deep neural network model with the highest accuracy to obtain the deep neural network model after the current round of optimization.

[0019] In some embodiments, in the optimization method of the deep neural network model, in the current iteration round, the simulator calculates the overall delay of the deep neural network model of at least one structure based on the delay dataset, and determines, based on the overall delay, a deep neural network model that satisfies the sub-target acceleration ratio of the current round and meets the preset accuracy conditions as the deep neural network model after the current round of optimization, including:

[0020] In the current iteration, the weight of each convolution kernel in the deep neural network model optimized in the previous round is calculated; the weight of the convolution kernel represents the degree of influence of the convolution kernel on the accuracy of the deep neural network model;

[0021] Based on the weight of each convolution kernel in the deep neural network model and the sub-target speedup ratio of the current iteration round, as well as the sub-target speedup ratio of the current round and the satisfaction of the preset accuracy conditions, the redundant convolution kernels of the current iteration round are determined at one time on a global scale;

[0022] The redundant convolution kernels are deleted by hiding them with binary masks, and a deep neural network model whose overall delay satisfies the sub-target acceleration ratio of the current round and meets the preset accuracy conditions is obtained as the deep neural network model after this round of optimization. At the same time, the convolution kernels hidden by the binary mask still participate in the next iteration round based on their weights.

[0023] In some embodiments, in the optimization method of the deep neural network model, the optimization method further includes:

[0024] The deep neural network model output by the latest preset number of iterations is retained as the rollback model in the rollback model pool;

[0025] If the model accuracy of the deep neural network model output by this round of iteration does not meet the preset accuracy conditions, the rollback model with the highest accuracy in the rollback model pool is selected as the rollback target model;

[0026] The iterative optimization of this round is performed again based on the rollback target model.

[0027] In some embodiments, in the optimization method of the deep neural network model, collecting the latency data of the convolution layer with different numbers of convolution kernels on the target platform, and obtaining a latency dataset including the latency data of each convolution layer with different numbers of convolution kernels, includes:

[0028] Collecting multiple candidate delay data of the convolution layer with different numbers of convolution kernels on the target platform;

[0029] The minimum candidate delay data corresponding to each number of convolution kernels is selected as the delay data of the convolution layer, and the delay data of the convolution layer with different numbers of convolution kernels is obtained.

[0030] In some embodiments, the method for optimizing the deep neural network model, the time delay data of the convolution layer with different numbers of convolution kernels on the target platform is collected, including:

[0031] determining a target operator matching the type of the target platform from a plurality of types of operators pre-configured in the simulator based on the type of the target platform; wherein the type of the target platform includes CPU, GPU, NPU, and FPGA; and the plurality of types of operators pre-configured in the simulator include CPU operator, GPU operator, NPU operator, and FPGA operator;

[0032] collecting the time delay data of the convolution layer with different numbers of convolution kernels on the target platform by using the simulator of the target operator.

[0033] In some embodiments, a device for optimizing a deep neural network model is also provided, and the device includes:

[0034] an output module configured to extract each convolution layer of the deep neural network model respectively and output the extracted convolution layer on the target platform respectively;

[0035] a collection module configured to change the number of convolution kernels of each convolution layer respectively, collect the time delay data of the convolution layer with different numbers of convolution kernels on the target platform respectively, and obtain a time delay data set including the time delay data of each convolution layer with different numbers of convolution kernels;

[0036] a calculation module configured to calculate the overall time delay of a plurality of deep neural network models with different structures respectively by using the simulator based on the time delay data set, and determine a deep neural network model with overall time delay meeting a target time delay condition; wherein the number of convolution kernels in at least one convolution layer of the deep neural network model with different structures is different;

[0037] a training module configured to retrain the selected deep neural network model to obtain a target deep neural network model dedicated to the target platform.

[0038] In some embodiments, an electronic device is also provided, which includes a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor communicates with the memory through the bus, and the machine readable instructions are executed by the processor to perform the steps of the method for optimizing the deep neural network model.

[0039] In some embodiments, a computer readable storage medium is also provided, and the computer readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the optimization method of the deep neural network model.

[0040] The application provides an optimization method, device, electronic equipment and medium for a deep neural network model. The optimization method extracts each convolutional layer of the deep neural network model, and separately outputs the extracted convolutional layers on a target platform. For each convolutional layer, the number of convolutional kernels of the convolutional layer is changed, the time delay data of the convolutional layer with different numbers of convolutional kernels on the target platform is collected, and a time delay data set including the time delay data of each convolutional layer with different numbers of convolutional kernels is obtained. Based on the time delay data set, the overall time delay of deep neural network models with different structures is calculated by a simulator, and a deep neural network model with an overall time delay meeting a target time delay condition is determined. At least one convolutional layer in the deep neural network model with different structures has different convolutional kernels. The selected deep neural network model is retrained to obtain a target deep neural network model dedicated to the target platform. The optimization method is based on a platform simulator that collects data on a real backend hardware platform and simulates the time delay of a deep neural network model that has been optimized (redundant convolutional kernels are removed). These simulated time delay data provide customized optimization basis for specific platforms for subsequent platform-aware deep learning model optimization. Based on the simulated time delay data, a deep neural network model with the highest accuracy and meeting a target speedup ratio on the target platform is generated, and the entire process does not need to master the specific hardware feature information of the platform. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0042] Figure 1 A method flowchart of the optimization method of the deep neural network model according to the embodiments of the application is shown;

[0043] Figure 2 A time delay data diagram of each convolutional layer collected according to the embodiments of the application is shown;

[0044] Figure 3 A method flowchart of determining a deep neural network model with an overall time delay meeting a target time delay condition according to the embodiments of the application is shown;

[0045] Figure 4 A method flow chart of the layer-by-layer model pruning method described in the present application is shown;

[0046] Figure 5 A comparison result of the exclusive optimal model generated on the Mobile CPU platform and other three neural network models described in the present application is shown;

[0047] Figure 6 A comparison result of the exclusive optimal model generated by the optimization method described in the embodiment of the present application and the original MobileNetV2 model in the number of convolution kernels of each layer is shown;

[0048] Figure 7 A comparison result of the exclusive optimal model generated on the NPU accelerator and other three neural network models described in the embodiment of the present application is shown;

[0049] Figure 8 A comparison result of the exclusive optimal model generated by the optimization method described in the embodiment of the present application and the original MobileNetV2 model in the number of convolution kernels of each layer is shown;

[0050] Figure 9 A comparison result of the exclusive optimal model generated on the FPGA platform and other three neural network models described in the embodiment of the present application is shown;

[0051] Figure 10 A comparison result of the exclusive optimal model generated by the optimization method described in the embodiment of the present application and the original MobileNetV2 model in the number of convolution kernels of each layer is shown;

[0052] Figure 11 A structure schematic diagram of the deep neural network model optimization device described in the embodiment of the present application is shown;

[0053] Figure 12 A structure schematic diagram of the electronic device described in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowchart used in the present application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowchart or one or more operations can be removed from the flowchart under the guidance of the content of the present application.

[0055] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0056] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0057] Deep learning based on deep neural networks (DNN) has achieved breakthrough results in many real-world applications. At the same time, due to the characteristics of data-driven and computationally intensive algorithms, deep learning algorithms usually require strong storage and computing power support to complete. The high complexity of deep neural network algorithm models makes their requirements for computing and storage resources very demanding. Especially on resource-constrained embedded devices, even if the general deep neural network model can be deployed, it is difficult to meet the requirements of speed and latency in actual applications. At the same time, on the other hand, neither the cloud nor the edge has unlimited resources. Under the premise of resource constraints, how to optimize the deep neural network runtime itself to make it more efficient is also a topic of great concern.

[0058] The prior art has been trying to solve the demanding requirements of deep learning on computing and storage resources from both hardware and software perspectives. On the one hand, as the backend hardware platform carrying the algorithm, various general-purpose and special-purpose architecture innovations have sprung up, from new high-performance GPU architectures and flexible scalable FPGA acceleration cluster systems at the data center level, to special accelerator cores integrated into SoCs on edge mobile devices, and so on. On the other hand, how to optimize the deep neural network runtime itself to make it more efficient has also achieved fruitful results. Various innovations focusing on the structure of neural network models have emerged in an endless stream, especially a series of slim "thin" neural network models such as MobileNets, SqueezeNet, etc. for edge mobile platforms. The emergence of these models has made it possible to deploy deep neural networks directly on mobile devices, promoting deep learning from the cloud to the edge. Secondly, optimization methods for existing models have also achieved great success, from neural network pruning, sparsification, to model quantization and matrix decomposition, etc. These technologies have greatly reduced the computational and storage overhead of deep neural networks, improving the speed of inference or training.

[0059] Now although there are various deep neural network optimization techniques and various new deep neural network field-specific hardware accelerator platforms. However, few works can combine the two organically. Current advanced neural network optimization techniques are mostly algorithm model level optimizations. Since the hardware architecture features of different platforms are quite different, and these optimization techniques cannot "perceive" these feature information, ultimately leading to their inability to achieve the expected optimization effect on all platforms.

[0060] Based on this, the embodiment of the application proposes a deep neural network model optimization method, device, electronic equipment and medium, the optimization method respectively extracts each convolution layer of the deep neural network model, and respectively outputs the extracted convolution layer on the target platform; for each convolution layer, the number of convolution kernels of the convolution layer is changed, the time delay data of the convolution layer with different numbers of convolution kernels is collected respectively, and a time delay data set including the time delay data of each convolution layer with different numbers of convolution kernels is obtained; based on the time delay data set, the overall time delay of deep neural network models with multiple different structures is calculated respectively by a simulator, and a deep neural network model with overall time delay meeting a target time delay condition is determined; wherein the deep neural network models with different structures are different in at least one convolution kernel in the convolution layer; the selected deep neural network model is retrained to obtain a target deep neural network model dedicated to the target platform; the optimization method is based on a platform simulator, which collects data on a real backend hardware platform and simulates the time delay of a deep neural network model after optimization (reducing redundant convolution kernels), the simulated time delay data provides customized optimization basis for specific platforms for subsequent platform-aware deep learning model optimization, and based on the simulated time delay data, a deep neural network model with the highest accuracy and meeting the target acceleration ratio on the target platform is generated, and the whole process does not need to master the specific hardware feature information of the platform.

[0061] Please refer to Figure 1 , Figure 1 The method flowchart of the deep neural network model optimization method described in the embodiment of the application is shown; specifically, the optimization method includes the following steps S101-S104:

[0062] S101, respectively extract each convolution layer of the deep neural network model, and respectively output the extracted convolution layer on the target platform;

[0063] S102, for each convolution layer, the number of convolution kernels of the convolution layer is changed, the time delay data of the convolution layer with different numbers of convolution kernels is collected respectively, and a time delay data set including the time delay data of each convolution layer with different numbers of convolution kernels is obtained;

[0064] S103, based on the time delay data set, the overall time delay of deep neural network models with multiple different structures is calculated respectively by a simulator, and a deep neural network model with overall time delay meeting a target time delay condition is determined; wherein the deep neural network models with different structures are different in at least one convolution kernel in the convolution layer;

[0065] S104, retrain the selected deep neural network model to obtain a target deep neural network model dedicated to the target platform.

[0066] The optimization method of the deep neural network model described in the present application focuses on a scalable platform simulator, which can collect data on different platforms and simulate the operation delay of a certain deep neural network model on the specific platform, and uses the method of deleting redundant convolution kernels to optimize the model. This method has more direct and significant acceleration effect than the method of deleting some weights alone (sparsification), and can iteratively optimize the model according to the delay data simulated by the platform simulator on the target platform, finally meet the expected optimization target, and unlike the current advanced general neural network optimization technology, the optimization method can automatically generate an exclusive optimal model for different platforms without mastering the hardware characteristics of the platform itself. The optimal model for different platforms generated has different network structures, ensuring the best optimization effect on each platform.

[0067] In the step S101, each convolution layer of the deep neural network model is extracted respectively, and the extracted convolution layer is output separately on the target platform.

[0068] For deep neural networks, the huge computational complexity will bring two problems to be solved urgently to the actual deployment of the neural network model. First, the model inference process will become slow due to the high computational complexity, which makes it difficult to deploy in real-time scenarios. Second, the intensive computing requirement will consume a large amount of power consumption, which greatly limits the resource-constrained embedded devices.

[0069] Neural network model pruning is an optimization method of deep neural network model. Based on the assumption that there is a large degree of redundancy in the weights of neural network, pruning of neural network model is to delete the unimportant weight parameters in the model. Under this kind of pruning method, the weight parameters of the whole neural network model will become very sparse. This sparsification will bring two great benefits to the deployment and operation of the whole deep neural network. First, the sparse weight after pruning can reduce the storage overhead through the compressed sparse matrix storage format such as CSR / CSC. On the other hand, the computational complexity of the whole neural network can be reduced because the related calculation of the deleted weights can be ignored.

[0070] According to the granularity of pruning, pruning techniques can be divided into five types: fine-grained pruning, vector-level pruning, kernel-level pruning, group pruning, and whole convolution kernel pruning. The optimization method described in the embodiments of the present application is mainly aimed at kernel-level pruning. Kernel-level pruning directly deletes redundant convolution kernels in the neural network, which can reduce the number of input and output channels in each layer of the model. For example, the first convolution kernel is completely pruned, and the corresponding output channel is one less. Therefore, pruning of convolution kernels can more efficiently accelerate deep learning applications. In the prior art, many works use pruning techniques to optimize neural networks. These works prune (zero out) redundant weights to sparsify the convolution kernels in the model. However, not all platforms can well support this sparse data structure, so these methods cannot guarantee good acceleration results on all platforms. Other works, on the contrary, use the method of deleting entire convolution kernels, which directly reduces the amount of computation and can achieve more significant acceleration. However, the disadvantage of these works is non-automation, i.e., the number of pruned convolution kernels needs to be manually determined, because different optimization choices may exist for different backend platforms. The above works cannot perceive the backend hardware, i.e., their optimization methods cannot guarantee the expected results on any platform and cannot make customized optimizations for specific hardware.

[0071] Each convolution layer of the deep neural network model is extracted and output separately on the target platform, which is realized based on the extensible platform simulator described in the embodiments of the present application. Only based on the real hardware platform latency data collected by the simulator to optimize the model, the optimization method described in the embodiments of the present application can realize "platform perception", and at the same time, can provide customized optimization basis for subsequent optimization of deep neural network models for target platforms, so as to purposefully optimize the model to achieve the target speedup ratio requirement.

[0072] Here, the deep neural network model is a pre-trained deep neural network model, each convolution layer in the pre-trained deep neural network model is extracted and deployed separately on the corresponding backend platform.

[0073] Here, the backend platform includes CPU, GPU, NPU, FPGA, etc.

[0074] For traditional CPUs and GPUs, directly use the operator API in common deep learning frameworks (TensorFlow, Caffe, etc.) to directly deploy the convolution layer on the target device.

[0075] Taking TensorFlow as an example, kernel operators in a basic operator library tf.nn of a deep neural network provided by the framework are called, and the kernel operators are also final calling implementations of operations in a convolution layer. A core thereof is tf.nn.conv2d, which is used to calculate a 2-dimensional convolution operation, and a convolution result is calculated through a given 4-dimensional input (batch number, channel number, input length and input width) and a convolution kernel.

[0076] The platform simulator described in the embodiments of the present application is extended on a Cambrian deep learning dedicated accelerator NPU platform, so that the convolution layer of the deep neural network model can be deployed on the NPU platform, and the NPU and the traditional CPU and GPU platform use the same set of acquisition time delay data methods. The core of the extension work is to adapt the TensorFlow framework, and on the basis of the original support of CPU and GPU, the support for NPU is added. To achieve this, first, the running mechanism of the TensorFlow framework when performing a deep learning task is analyzed, and at the same time, the software call stack of the NPU platform is combined to develop and extend the TensorFlow framework.

[0077] The platform simulator described in the embodiments of the present application is extended on an FPGA, so that the convolution layer of the deep neural network model can be deployed on the FPGA platform. The flexibility and efficiency of the FPGA mainly reflect that it can generate a specific hardware structure according to the structure of the deep neural network model, perform fine-grained programming on the gate array, and generate an IP core (Overlay) for running the neural network model. Unlike NPU, because FPGA dedicated acceleration is to program the gate array according to the model structure, it cannot be deployed and implemented based on mainstream general-purpose frameworks such as TensorFlow. Here, each layer operator operation in the neural network model needs to be implemented separately and mapped to the FPGA to implement the corresponding gate array programming.

[0078] It should be noted that using a traditional HDL to implement the hardware corresponding to the neural network algorithm has a complexity that is difficult to imagine. Xilinx provides a Vivado high-level synthesis (HLS) tool, which can implement programming of FPGA programmable devices using C, C++ and SystemC language specifications without manually creating RTL, thereby accelerating IP creation.

[0079] In the step S102, for each convolution layer, the number of convolution kernels of the convolution layer is changed, the time delay data of the convolution layer with different numbers of convolution kernels on the target platform are collected respectively, and a time delay data set including the time delay data of each convolution layer with different numbers of convolution kernels is obtained.

[0080] Specifically, collecting the latency data of the convolution layer with different numbers of convolution kernels on the target platform includes:

[0081] Based on the type of the target platform, determine a target operator that matches the target platform type from a plurality of types of operators pre-configured in the simulator; wherein the types of the target platform include CPU, GPU, NPU, and FPGA; and the plurality of types of operators pre-configured in the simulator include CPU operators, GPU operators, NPU operators, and FPGA operators;

[0082] The delay data of the convolution layer with different numbers of convolution kernels on the target platform is collected by using a simulator of the target operator.

[0083] Here, the CPU operators and GPU operators are already available on traditional CPU and GPU platforms. The NPU operators are implemented by extending and extending mainstream general frameworks such as TensorFlow, while the FPGA operators are implemented by programming FPGA programmable devices separately.

[0084] Different numbers of convolutional kernels in a convolutional layer result in different numbers of input and output channels. Therefore, during the data collection step, each convolutional layer in the pre-trained deep neural network model is extracted and deployed separately on the corresponding backend platform. Next, the input and output channel parameters for each layer are varied, allowing for the traversal of latency data for each layer under varying numbers of input and output channels.

[0085] Specifically, each layer of the neural network to be optimized is extracted and deployed to the backend platform. The convolution kernels of each layer are then removed one by one, and each layer is assigned a different number of input and output channels. This is because removing convolution kernels from a layer changes the number of input channels between that layer's output and the next layer's input channels. Simultaneously, latency data is collected for each layer in the model, regardless of the number of input and output channels (i.e., the number of convolution kernels).

[0086] For example, take the MobileNetV2 model as an example, please refer to Figure 2 , Figure 2 A schematic diagram of the delay data of each convolutional layer collected in an embodiment of the present application is shown.

[0087] When deploying data collection, taking the optimization of MobileNetV2 deep neural network model as an example, the goal is to collect delay data and generate Figure 2The data table in the data table. Because there may be errors in the actual hardware platform running time, the operation process will be disturbed by other tasks in the system, so that the time delay is large. Therefore, in the embodiment of the application, two groups of data tables are continuously collected, and the smaller data in the same position of the two groups of tables is taken as the time delay data of the layer operation under the corresponding input and output channel number, so as to avoid errors.

[0088] Specifically, in the embodiment of the application, the time delay data of the convolution layer with different convolution kernel numbers on the target platform is collected, and a time delay data set including the time delay data of each convolution layer with different convolution kernel numbers is obtained, including:

[0089] Collecting a plurality of candidate time delay data of the convolution layer with different convolution kernel numbers on the target platform;

[0090] Selecting the smallest candidate time delay data corresponding to each convolution kernel number as the time delay data of the convolution layer, and obtaining the time delay data of the convolution layer with different convolution kernel numbers.

[0091] In the step S103, based on the time delay data set, the overall time delay of a plurality of deep neural network models with different structures is calculated respectively by the simulator, and a deep neural network model whose overall time delay meets the target time delay condition is determined; wherein the convolution kernel of at least one convolution layer in the deep neural network model with different structures is different.

[0092] Here, the target time delay condition is determined based on the initial time delay data and the accuracy requirement of the deep neural network model in advance.

[0093] Here, the overall time delay of the plurality of deep neural network models with different structures is the deep neural network model after pruning. The number of convolution kernels of at least one convolution layer of the deep neural network model with different structures is different, that is, the input and output channel numbers of at least one convolution layer are different.

[0094] With the time delay data set, that is, Figure 2 The time delay data table shown in FIG. 2 can simulate the kernel operation time delay of a certain model after pruning (deleting redundant convolution kernels). Specifically, first, the number of remaining convolution kernels of each layer of the current model is analyzed, and then the input and output channel numbers of each convolution layer are obtained. At this time, the time delay data of each layer of the model under the corresponding input and output channel number is found from the previously collected data, such as the part framed in FIG. 2. Next, the time delay of each convolution layer corresponding to the input / output channel number is accumulated, as shown in the following formula (1), that is, the overall time delay of the deep neural network model can be obtained.

[0095]

[0096] Wherein, T siman overall latency of the deep neural network model, i.e., a kernel operation latency of the deep neural network model simulated by the simulator, t k,i,o represents a latency of the k-th layer of the collected model at the input channel number i and the output channel number o, and n represents the number of convolution layers of the deep neural network model.

[0097] By deploying each layer of the neural network on a specific backend platform and traversing different input and output channel numbers, the latency data of each convolution layer of any kind of deep neural network at different input and output channel numbers can be collected on any platform. Then, the kernel operation latency of the model under various pruning conditions, i.e., each layer having different numbers of convolution kernels, can be simulated based on the data.

[0098] It should be noted that the data collection in step S102 can be completed in advance, because the core step of the platform simulator is independent of the optimization algorithm in step S103.

[0099] In the embodiments of the present application, the target latency condition is a total target speedup ratio. The total target speedup ratio is the ratio of the model operation latency before and after optimization, and can represent the optimization effect.

[0100] In the embodiments of the present application, please refer to Figure 3 Based on the latency data set, the overall latency of deep neural network models with multiple different structures is calculated by the simulator, and a deep neural network model whose overall latency meets the target latency condition is determined. The method comprises the following steps S301-S304:

[0101] S301, determining a total target speedup ratio according to the target accuracy requirement of the deep neural network model; the total target speedup ratio represents the performance improvement degree of the deep neural network model before and after optimization;

[0102] S302, performing multiple rounds of iterative optimization on the deep neural network model by the simulator, and determining a sub-target speedup ratio of each iteration round according to the iteration round and the total target speedup ratio;

[0103] S303, in the current iteration round, the simulator calculates the overall latency of a deep neural network model with at least one structure based on the latency data set, and determines a deep neural network model that meets the sub-target speedup ratio of the current round and satisfies a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall latency;

[0104] S304, performing the next round of iterative optimization on the deep neural network model after the current round of optimization until the overall latency of the deep neural network model meets the total target speedup ratio.

[0105] That is, in the embodiments of the present application, a progressive method is used to optimize the model to meet the target speedup. First, the entire optimization process is divided into N steps, that is, it is realized by N rounds of iteration progression, where N is one of the important hyperparameters of the optimization algorithm. In this way, there will be a sub-target speedup in each round of iteration. Specifically, by learning from the learning rate decay idea in neural network training, decay is set for each round of iteration sub-target to prevent some important convolution kernels from being mistakenly deleted in the later optimization, resulting in too high precision loss. Therefore, the sub-target speedup T subi as shown in formula (2), where T init is the initial target speedup without considering decay. In this way, the final total target speedup T final is shown in formula (3). In fact, when setting T init , the final total target speedup T final needs to be determined first, and then the initial target T init is back calculated according to decay and N.

[0106]

[0107]

[0108] where T init represents the initial target speedup without considering decay, T subi is the sub-target speedup of the i-th round of iteration, d i represents the decay factor for the i-th round of iteration; and T final is the final total target speedup.

[0109] In each round of iteration, according to the delay simulated by the platform simulator, the model is pruned (redundant convolution kernels are deleted) until the sub-target speedup T subi of the round of iteration is met, and the pruned model is input into the next round of iteration. When the last round of iteration is completed, that is, all sub-targets are met, a new model that meets the final total target speedup T final is obtained, and then the model is retrained to obtain the final optimal model.

[0110] Based on the platform simulator, all data can be collected in advance, and the optimization process only needs to call the data to simulate the model delay, so the optimization algorithm does not need to be deployed on the target platform, and the optimization process does not need to interact with the backend platform. The core step "pruning" in the algorithm is to prune the model according to the sub-target of each round of iteration and the simulated delay, so as to meet the sub-target speedup of each round of iteration. Because different backend hardware platforms have different delay data characteristics, the final optimal model generated is platform-specific.

[0111] Therefore, in step S104, the screened deep neural network model is retrained to obtain a target deep neural network model that is specific to the target platform.

[0112] That is, the screened deep neural network model, i.e., a new model that meets the final total target speedup ratio T final

[0113] In the embodiments of the present application, two specific pruning methods are provided for the model pruning performed in step S103.

[0114] First, the layer-by-layer model pruning method. Specifically, in the current iteration round, the simulator calculates the overall latency of deep neural network models of multiple different structures based on the latency dataset, and determines a deep neural network model that meets the sub-target speedup ratio of the current round and satisfies the preset accuracy condition as the deep neural network model optimized in the current round based on the overall latency; comprising:

[0115] The simulator traverses the k prunable convolutional layers in the deep neural network model. For each convolutional layer, the number of convolutional kernels that need to be retained in the convolutional layer is determined to meet the sub-target speedup ratio of the current iteration round;

[0116] Based on the influence degree of different convolutional kernels in the convolutional layer on the accuracy of the deep neural network model, the convolutional kernels to be retained are determined to obtain a candidate deep neural network model corresponding to the convolutional layer that meets the preset accuracy condition;

[0117] From the k candidate deep neural network models corresponding to the k prunable convolutional layers, the deep neural network model with the highest accuracy is screened out to obtain the deep neural network model optimized in the current round.

[0118] The layer-by-layer model pruning method described in the embodiments of the present application strives to ensure that the accuracy loss of each pruning in each round of optimization is minimized. In the N-iteration progressive optimization algorithm, each iteration reduces the redundant convolutional kernels in the model according to the simulated platform latency to meet the sub-target speedup ratio of the current iteration.

[0119] Please refer to Figure 4 , Figure 4 The method flowchart of the layer-by-layer model pruning method described in the present application is shown. Specifically, as Figure 4 ​As shown, for a deep neural network model with k layers to be pruned, in one iteration, k layers in the model are traversed. For each layer, the number of convolution kernels in the layer is reduced one by one according to the simulated latency dataset until the simulated model latency meets the sub-target speedup ratio of the current iteration. It should be noted that when the number of convolution kernels in a layer changes, the number of output channels of the layer and the number of input channels of the next layer also change accordingly. When the number of convolution kernels that need to be retained in the current layer to meet the sub-target of the current iteration is known, the convolution kernels to be retained are determined based on the influence of different convolution kernels in the convolution layer on the accuracy of the deep neural network model. In the embodiment of the application, the maximum L2 norm is used to select the convolution kernels to be retained.

[0120] After the pruning of each convolution layer is completed, the pruned model of the layer needs to be fine-tuned to restore the accuracy, and a candidate deep neural network model is generated, and then the next layer pruning process is entered. Therefore, k candidate deep neural network models meeting the sub-target speedup ratio are obtained in each iteration cycle, k is the number of layers of the model, and each candidate model has only one layer pruned. In one iteration, after traversing all the prunable layers, the model with the highest accuracy is selected from the k candidate models as the final result of the pruning of the current iteration, and is used for the next iteration. In summary, in each iteration cycle, only the layer that causes the least accuracy loss is pruned, while the sub-target speedup ratio and the preset accuracy condition are guaranteed.

[0121] It should be noted that this is not the only layer-by-layer pruning method. Other methods can be considered when determining which convolution kernels to retain in each layer. Even the entire pruning process can consider other methods such as reinforcement learning, for example, starting from more fine-grained weight sparsification, etc.

[0122] In the embodiment of the application, another global model pruning method is also performed when step S103 is performed. The layer-by-layer model pruning method described in the embodiment of the application has two major drawbacks. First, using the layer-by-layer pruning method to optimize the convolutional neural network is relatively inefficient, because after pruning each layer in each iteration, the accuracy needs to be restored by retraining, i.e., k times of retraining are required for each iteration optimization, as shown in the following table. Figure 4The method of pruning the model layer by layer ignores an important factor that the "importance" of different convolution kernels in each layer changes with the process of pruning and retraining. Specifically, the L2 norm of each convolution kernel changes with the training. Assuming that the pruned convolution kernel continues to participate in the training, it is possible that the L2 norm increases in the later training, i.e., the importance becomes stronger. Therefore, this layer-by-layer pruning method may mistakenly prune the potentially important convolution kernel, and such operation cannot be undone. In this case, the later process is equivalent to completely retraining the remaining convolution kernel to replace the important convolution kernel pruned in the early stage.

[0123] Based on this, the method for optimizing a deep neural network model described in the embodiments of the present application, in the current iteration round, the simulator calculates the overall latency of the deep neural network model of at least one structure based on the latency data set, and determines the deep neural network model that meets the sub-target speedup ratio of the current round and the preset accuracy condition as the deep neural network model after the optimization of the current round based on the overall latency, comprising:

[0124] In the current iteration round, the importance score of each convolution kernel in the deep neural network model optimized in the last round is calculated; the importance score of the convolution kernel represents the degree of influence of the convolution kernel on the accuracy of the deep neural network model;

[0125] Based on the importance score of each convolution kernel in the deep neural network model, and the sub-target speedup ratio of the current round and the preset accuracy condition, the redundant convolution kernel of the current iteration round is determined globally at one time;

[0126] The redundant convolution kernel is deleted in the manner of hiding the redundant convolution kernel with a binary mask, and the deep neural network model whose overall latency meets the sub-target speedup ratio of the current round and the preset accuracy condition is obtained as the deep neural network model after the optimization of the current round, while the convolution kernel hidden by the binary mask still participates in the next iteration round based on its weight.

[0127] Specifically, based on the importance score of each convolution kernel in the deep neural network model, and the sub-target speedup ratio of the current round and the preset accuracy condition, the redundant convolution kernel of the current iteration round is determined globally at one time; comprising:

[0128] Based on the sub-target speedup ratio of the current round and the preset accuracy condition, the target number of redundant convolution kernels is determined;

[0129] The target number of convolution kernels with the lowest importance score is determined as the redundant convolution kernel globally at one time.

[0130] The global model pruning method can globally evaluate the importance of each convolution kernel during optimization, and can recover important convolution kernels that have been pruned previously. Specifically, in the embodiments of the present application, a binary mask method is adopted to delete redundant convolution kernels in the global range at a time during each round of iterative optimization, that is, the mask is set to 0 to meet the sub-objective of the current round of iteration. Here, the mask also serves as a quantitative standard for the importance of the convolution kernel. The importance of each convolution kernel is evaluated by an evaluation function based on the change amount of the global loss function, with 0 being redundant and 1 being important. In this way, only one retraining is needed after global pruning in each round of iterative optimization, greatly improving the optimization efficiency. The pruned convolution kernels still participate in subsequent optimization under the mask and may be reactivated in the subsequent optimization process, while the convolution kernels that become unimportant during training will be deleted.

[0131] The global dynamic model pruning method has the following two main advantages: first, the convolution kernel importance evaluation based on the change amount of the network loss function is global, and the important convolution kernel reduction is also global rather than layer-by-layer. This means that there is only one pruning and retraining process in each round of iterative optimization. Compared with the retraining after pruning each layer in the layer-by-layer pruning, the global pruning greatly shortens the time consumption and overall training overhead of the entire optimization scheme. Second, the dynamic training based on the importance mask enables the optimization process to be more coherent and meaningful. The "importance" of different convolution kernels in each layer changes dynamically with the pruning and retraining process. The convolution kernels that were pruned earlier may become important as the optimization process progresses. In the layer-by-layer pruning process, the pruned convolution kernels cannot be recovered, so the remaining convolution kernels need to be retrained to replace the important convolution kernels that were pruned, which is inefficient. In contrast, in the dynamic training based on the importance mask, the pruned convolution kernels still participate in subsequent optimization under the mask. Moreover, this method can timely evaluate the importance of all convolution kernels, thereby recovering important convolution kernels that were mistakenly deleted earlier.

[0132] Specifically, in the embodiments of the present application, in order to globally and dynamically reduce redundant convolution kernels, thereby reducing the training overhead caused by layer-by-layer reduction, and at the same time recovering some important convolution kernels that have been reduced during the optimization process, a binary mask mechanism is introduced. This mechanism can temporarily exclude globally redundant convolution kernels (mask = 0), and can also dynamically recover (mask = 1) some convolution kernels that were mistakenly deleted earlier according to the change in the importance of each convolution kernel during optimization and training.

[0133] Specifically, the mask of the convolution kernel is determined by the following formula (4).

[0134]

[0135] wherein, m q a mask h q (W * ) represents a function for calculating the mask of the convolution kernel q according to the weight matrix W* of the convolution kernel; the weight matrix W* of the convolution kernel includes the weight of each convolution kernel; a priority C l The first L convolution kernels are sorted in descending order; wherein, the priority C l is calculated based on the weight matrix W* of the convolution kernel, and specifically, the influence degree of a single convolution kernel on the final loss function output value of the entire network is calculated as the priority C l of the giant convolution kernel based on the weight matrix W, and β represents a coefficient.

[0136] The above formula (4) generates a binary mask for each convolution kernel according to the number L of convolution kernels to be retained determined in the current round of optimization, that is, the mask of the first L convolution kernels with high importance is 1, and the mask of the other convolution kernels is 0.

[0137] Here, the priority C l of the convolution kernel is a similar concept to the importance score, in some embodiments, the priority C l is the importance score, and in some embodiments, the priority C l is obtained by transforming the importance score.

[0138] In the embodiments of the present application, the priority C l of the convolution kernel is calculated by the following method: calculating the loss function change after deleting the convolution kernel, and taking the loss function change as the priority C l of the convolution kernel, that is, the importance score.

[0139] The loss function of the modified deep neural network model is still non-convex, and its optimization method is slightly different from the traditional neural network training method. First, before each training, the weights W of all the current convolution kernels need to be re-evaluated, and the function h q (W *) to update the mask m of the convolution kernel. Next, similar to the traditional neural network training, the weights W of the convolution kernel are updated based on the back propagation algorithm. In the gradient descent calculation, the convolution kernel that has been deleted still participates in the subsequent training under mask = 0, and here the partial derivative of all convolution kernels under the mask is required when calculating the gradient. At the same time, it needs to be noted that here all the weights W of the convolution kernel are updated, not only the currently retained convolution kernel. Under this optimization training method, the global dynamic optimization method described in the embodiments of the present application can timely evaluate the importance change of the convolution kernel after the last training before each training, so as to restore the convolution kernel with high importance in this optimization.

[0140] Since the number of pruned convolution kernels in the global pruning process is also derived from the platform simulator simulation delay to ensure the sub-goal of each round of optimization, the above-mentioned global dynamic model pruning method is used in each round of iterative optimization, and after all iterations are completed, the generated model needs to be retrained for a long time as the layer-by-layer pruning method, and the finally generated model is also a platform-specific optimal model that meets the total acceleration target and has a certain guaranteed accuracy.

[0141] In the optimization method of the deep neural network model described in the embodiments of the present application, the total target acceleration ratio is determined according to the target accuracy requirement of the deep neural network model. The total target acceleration ratio proposed here is obtained after balancing the model accuracy after optimization. If the target acceleration ratio is set too high, the final model accuracy may be lost too much. To solve the above problem, the optimization method described in the embodiments of the present application also designs a rollback strategy when the optimization fails.

[0142] The rollback strategy is as follows: when the optimization is performed to a certain round of iteration, if the output model accuracy of the round of iteration is lower than a threshold, the rollback mechanism is triggered. Under the rollback mechanism, the output model of each round of iteration is saved to a rollback model pool. It needs to be noted that since the output model of each round of iteration is fine-tuned, it may fall into the "saddle point" of the neural network training, that is, it cannot be guaranteed that the model with high accuracy after a short time of fine-tuning can achieve significantly higher accuracy in the retraining stage. Therefore, the model pool is set to accommodate the latest preset number of iterations of the optimized deep neural network model, for example, 5 models obtained by 5 iterations. When the rollback mechanism is triggered, the 5 models in the pool are fine-tuned for a short time, and the model with the highest accuracy among the 5 models is selected as the rollback target. After determining the rollback target, the optimization process is terminated, and the rolled back model is retrained to obtain the final optimal model, and the actual acceleration ratio is output.

[0143] That is, the optimization method of the deep neural network model described in the embodiments of the present application also includes:

[0144] reserve the latest preset number of iteration outputs of the deep neural network model as rollback models in a rollback model pool as rollback models;

[0145] When the model accuracy of the deep neural network model of the current iteration output does not meet the preset accuracy condition, the rollback model with the highest accuracy in the rollback model pool is selected as the rollback target model;

[0146] Based on the rollback target model, the iteration optimization of the current round is re-performed.

[0147] Here, the preset accuracy condition is a preset accuracy threshold, and the model accuracy of the optimized deep neural network model does not meet the preset accuracy condition, that is, the model accuracy of the deep neural network model output in this round of iteration is lower than the preset accuracy threshold.

[0148] In the embodiments of the present application, an exemplary state-of-the-art mobile terminal simplified model MobileNetV2 is selected as the optimization object, and the optimal model dedicated to the Mobile CPU, NPU accelerator and FPGA platforms is generated respectively, and the latency of the original MobileNetV2 on different platforms (Mobile CPU, NPU, FPGA) is compared to verify the optimization effect.

[0149] For the optimization result of the Mobile CPU platform, the present application embodiment designs two groups of experiments with different target acceleration ratios. Here, the target deep neural network model dedicated to the target platform obtained by optimization is referred to as the dedicated optimal model.

[0150] Please refer to Figure 5 , Figure 5 The comparison results of the dedicated optimal model generated on the Mobile CPU platform and the other three neural network models are shown. The dedicated optimal model (CompactNet) obtained by the optimization method of the present application embodiment can achieve a maximum of 1.8x acceleration ratio without losing accuracy compared with the original MobileNetV2, and if the target is set to 1.5x acceleration, the final optimal model will be slightly higher in accuracy than the original model. Compared with the current advanced pruning-based neural network optimization methods NetAdapt, MorphNet and ADC, the optimization method of the present application embodiment can exceed these works in acceleration ratio and accuracy on the same ImageNet dataset. Compared with the original MobileNetV2, the number of convolution kernels in the optimal model dedicated to the Mobile CPU obtained by the optimization method of the present application embodiment is as shown in Figure 6 , Figure 6The comparison results of the number of convolution kernels of each layer of the Mobile CPU-specific optimal model generated by the optimization method of the embodiments of the present application and the original MobileNetV2 model are shown.

[0151] For the NPU accelerator, Figure 7 The comparison results of the specific optimal model generated on the NPU accelerator and the other three neural network models are shown; according to Figure 7 It can be known that, since the other comparative models cannot perceive the backend platform, the models output by these general optimization methods (NetAdapt, MorphNet and ADC) cannot achieve the same optimization effect on the NPU accelerator as on the Mobile CPU. On the contrary, the optimization method (CompactNet) described in the embodiments of the present application can still generate a specific optimal model on the NPU accelerator. Please refer to Figure 8 , Figure 8 The comparison results of the number of convolution kernels of each layer of the NPU accelerator-specific optimal model generated by the optimization method of the embodiments of the present application and the original MobileNetV2 model are shown.

[0152] For the FPGA platform, Figure 9 The comparison results of the specific optimal model generated on the FPGA platform and the other three neural network models are shown; similar to the NPU accelerator platform, the current advanced deep neural network optimization technology cannot make targeted optimization for the characteristics of the FPGA platform. The network models generated by these more general optimization techniques cannot achieve the best optimization effect on the FPGA. However, the CompactNet platform-aware optimization method proposed in the embodiments of the present application can still make targeted pruning optimization for the target MobileNetV2 network based on the simulation delay data of the FPGA platform simulator, to generate a FPGA platform-specific optimal model. Such specific optimization results are shown in Figure 10 It can be seen that, whether it is the speedup ratio or the accuracy, the existing optimization techniques (NetAdapt, MorphNet and ADC) cannot perceive the platform. Please refer to Figure 10 , Figure 10 The comparison results of the number of convolution kernels of each layer of the NPU accelerator-specific optimal model generated by the optimization method of the embodiments of the present application and the original MobileNetV2 model are shown.

[0153] The above experiment proves that the optimal model generated by the embodiment of the application is platform-specific, that is, the model cannot achieve the expected optimization effect after being separated from its target backend platform. The root cause of this phenomenon lies in the great difference in the deep architecture characteristics of different backend platforms. The experiment proves that the CompactNet optimization method described in the embodiment of the application can generate a platform-specific optimal model, and the performance in terms of latency and accuracy is superior to the current advanced deep neural network optimization work, the acceleration effect is more obvious at the same accuracy level, and the optimization process takes less time.

[0154] Based on the same inventive concept, the embodiment of the application also provides a deep neural network model optimization device corresponding to the deep neural network model optimization method. Since the principle of solving problems in the device of the embodiment of the application is similar to the deep neural network model optimization method described above, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described.

[0155] Please refer to Figure 11 , Figure 11 The structure of the deep neural network model optimization device described in the embodiment of the application is shown, specifically, the device comprises:

[0156] The output module 1101 is configured to extract each convolutional layer of the deep neural network model respectively, and output the extracted convolutional layer on the target platform separately;

[0157] The collection module 1102 is configured to, for each convolutional layer, change the number of convolutional kernels of the convolutional layer in a traversal manner, collect the latency data of the convolutional layer with different numbers of convolutional kernels on the target platform respectively, and obtain a latency data set comprising the latency data of each convolutional layer with different numbers of convolutional kernels;

[0158] The calculation module 1103 is configured to calculate the overall latency of deep neural network models with multiple different structures respectively based on the latency data set through the simulator, and determine a deep neural network model whose overall latency meets the target latency condition; wherein the number of convolutional kernels in at least one convolutional layer of the deep neural network models with different structures is different.

[0159] The training module 1104 is configured to retrain the selected deep neural network model to obtain a target deep neural network model specific to the target platform.

[0160] Based on this, the embodiment of the present application proposes an optimization device of a deep neural network model, each convolutional layer of the deep neural network model is extracted respectively, and the extracted convolutional layers are respectively output separately on a target platform; for each convolutional layer, the number of convolution kernels of the convolutional layer is changed in a traversal manner, the time delay data of the convolutional layer with different numbers of convolution kernels on the target platform is collected respectively, and a time delay data set including the time delay data of each convolutional layer with different numbers of convolution kernels is obtained; based on the time delay data set, the overall time delay of deep neural network models with multiple different structures is calculated respectively by a simulator, and a deep neural network model with an overall time delay meeting a target time delay condition is determined; wherein the deep neural network models with different structures are different in at least one convolutional layer; the selected deep neural network model is retrained to obtain a target deep neural network model dedicated to the target platform; the optimization method is based on a platform simulator, which collects data on a real back-end hardware platform and simulates the time delay of a deep neural network model after optimization (redundant convolution kernels are deleted), the simulated time delay data provides a customized optimization basis for a specific platform for subsequent platform-aware deep learning model optimization, and based on the simulated time delay data, a deep neural network model with the highest accuracy and meeting the target speedup ratio on the target platform is generated, and the entire process does not need to master the specific hardware feature information of the platform itself.

[0161] In some embodiments, the optimization device of the deep neural network model, the calculation module, when calculating the overall time delay of deep neural network models with multiple different structures based on the time delay data set by the simulator and determining a deep neural network model with an overall time delay meeting a target time delay condition, is specifically used for:

[0162] determining a total target speedup ratio according to the target accuracy requirement of the deep neural network model; the total target speedup ratio represents the performance improvement degree of the deep neural network model before and after optimization;

[0163] performing multiple rounds of iterative optimization on the deep neural network model by the simulator, and determining a sub-target speedup ratio of each iteration round according to the iteration round and the total target speedup ratio;

[0164] in the current iteration round, the simulator calculates the overall time delay of a deep neural network model with at least one structure based on the time delay data set, and determines a deep neural network model meeting the sub-target speedup ratio of the current round and meeting a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall time delay;

[0165] performing the next round of iterative optimization on the deep neural network model after the current round of optimization until the overall time delay of the deep neural network model meets the total target speedup ratio.

[0166] In some embodiments, in the optimization device of the deep neural network model, the calculation module is specifically configured to, in the current iteration round, calculate the overall latency of the deep neural network model of each structure based on the latency data set by the simulator, and determine the deep neural network model satisfying the sub-target speedup ratio of the current round and the preset accuracy condition as the deep neural network model after the optimization of the current round based on the overall latency.

[0167] The simulator traverses the k prunable convolutional layers in the deep neural network model, determines the number of convolutional kernels that need to be retained for each convolutional layer to meet the sub-target speedup ratio of the current iteration round, and determines the convolutional kernel to be retained based on the influence degree of different convolutional kernels in the convolutional layer on the accuracy of the deep neural network model, to obtain the candidate deep neural network model of the convolutional layer meeting the preset accuracy condition.

[0168] The simulator traverses the k prunable convolutional layers in the deep neural network model, determines the number of convolutional kernels that need to be retained for each convolutional layer to meet the sub-target speedup ratio of the current iteration round, and determines the convolutional kernel to be retained based on the influence degree of different convolutional kernels in the convolutional layer on the accuracy of the deep neural network model, to obtain the candidate deep neural network model of the convolutional layer meeting the preset accuracy condition.

[0169] The simulator traverses the k prunable convolutional layers in the deep neural network model, determines the number of convolutional kernels that need to be retained for each convolutional layer to meet the sub-target speedup ratio of the current iteration round, and determines the convolutional kernel to be retained based on the influence degree of different convolutional kernels in the convolutional layer on the accuracy of the deep neural network model, to obtain the candidate deep neural network model of the convolutional layer meeting the preset accuracy condition.

[0170] In some embodiments, in the optimization device of the deep neural network model, the calculation module is specifically configured to, in the current iteration round, calculate the overall latency of the deep neural network model of each structure based on the latency data set by the simulator, and determine the deep neural network model satisfying the sub-target speedup ratio of the current round and the preset accuracy condition as the deep neural network model after the optimization of the current round based on the overall latency.

[0171] In the current iteration round, the importance score of each convolutional kernel in the deep neural network model after the optimization of the last round is calculated; the importance score of the convolutional kernel represents the influence degree of the convolutional kernel on the accuracy of the deep neural network model.

[0172] Based on the importance score of each convolutional kernel in the deep neural network model and the sub-target speedup ratio of the current round and the preset accuracy condition, the redundant convolutional kernel of the current iteration round is determined in the global range at one time.

[0173] The redundant convolutional kernel is deleted in the manner of hiding the redundant convolutional kernel by a binary mask, to obtain the deep neural network model satisfying the sub-target speedup ratio of the current round and the preset accuracy condition as the deep neural network model after the optimization of the current round, and meanwhile, the convolutional kernel hidden by the binary mask still participates in the next iteration round based on the weight thereof.

[0174] In some embodiments, the optimization device of the deep neural network model further comprises:

[0175] a rollback module configured to reserve the latest preset number of iteration output deep neural network models as rollback models in a rollback model pool;

[0176] when the model accuracy of the iteration output deep neural network model of the current round does not meet the preset accuracy condition, the rollback model with the highest accuracy in the rollback model pool is selected as a rollback target model;

[0177] re-performing the iteration optimization of the current round based on the rollback target model.

[0178] In some embodiments, the deep neural network model optimization device, the collection module, when collecting the time delay data of the convolution layer with different convolution kernel numbers on the target platform, obtains a time delay data set including the time delay data of each convolution kernel number of the convolution layer, and the time delay data set includes:

[0179] collecting a plurality of candidate time delay data of the convolution layer with different convolution kernel numbers on the target platform;

[0180] selecting the smallest candidate time delay data corresponding to each convolution kernel number as the time delay data of the convolution layer, and obtaining the time delay data of the convolution layer with different convolution kernel numbers.

[0181] In some embodiments, the deep neural network model optimization device, the collection module, when collecting the time delay data of the convolution layer with different convolution kernel numbers on the target platform, is specifically configured to:

[0182] based on the type of the target platform, determining a target operator matching the target platform type from a plurality of types of operators pre-configured by the simulator; wherein the type of the target platform includes CPU, GPU, NPU, and FPGA; and the plurality of types of operators pre-configured by the simulator include CPU operator, GPU operator, NPU operator, and FPGA operator;

[0183] collecting the time delay data of the convolution layer with different convolution kernel numbers on the target platform by using the simulator of the target operator.

[0184] Based on the same inventive concept, the embodiments of the present application also provide an electronic device corresponding to the deep neural network model optimization method. Since the principle of solving problems in the electronic device of the embodiments of the present application is similar to the above-mentioned deep neural network model optimization method of the embodiments of the present application, the implementation of the electronic device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0185] Please refer to Figure 12 , Figure 12A structural schematic diagram of the electronic device is shown; specifically, the electronic device 1200 includes a processor 1202, a memory 1201, and a bus, the memory 1201 stores machine readable instructions executable by the processor 1202, when the electronic device 1200 is running, the processor 1202 and the memory 1201 communicate through the bus, and the machine readable instructions are executed by the processor 1202 to perform the steps of the optimization method of the deep neural network model.

[0186] Based on the same inventive concept, the computer readable storage medium corresponding to the optimization method of the deep neural network model is also provided in the embodiments of the present application. Since the principle of solving problems in the computer readable storage medium in the embodiments of the present application is similar to the optimization method of the deep neural network model described above, the implementation of the computer readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described here.

[0187] A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is run by a processor to execute the steps of the optimization method of the deep neural network model.

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system and device can refer to the corresponding process in the method embodiment, which will not be described here in the present application. In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The above-described device embodiments are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some communication interface, indirect coupling or communication connection between the devices or modules, which can be electrical, mechanical or other forms.

[0189] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0190] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0191] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various media that can store program codes.

[0192] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of optimizing a deep neural network model, characterized by, The optimization method comprises: Respectively extract each convolutional layer of the deep neural network model, and separately output the extracted convolutional layer on a target platform; For each convolutional layer, the number of convolution kernels of the convolutional layer is changed, the time delay data of the convolutional layer with different numbers of convolution kernels on the target platform are respectively collected, and a time delay data set including the time delay data of each convolutional layer with different numbers of convolution kernels is obtained; collecting the time delay data of the convolutional layer with different numbers of convolution kernels on the target platform comprises: based on the type of the target platform, determining a target operator matching the type of the target platform from a plurality of types of operators pre-configured in the simulator; wherein the type of the target platform comprises CPU, GPU, NPU and FPGA; the plurality of types of operators pre-configured in the simulator comprise CPU operator, GPU operator, NPU operator and FPGA operator; and the time delay data of the convolutional layer with different numbers of convolution kernels on the target platform are collected by using the simulator of the target operator; Based on the time delay data set, the overall time delay of a plurality of deep neural network models with different structures is calculated by the simulator, and a deep neural network model with an overall time delay meeting a target time delay condition is determined; wherein the number of convolution kernels in at least one convolutional layer in the deep neural network model with different structures is different; The selected deep neural network model is retrained to obtain a target deep neural network model dedicated to the target platform; Based on the time delay data set, the overall time delay of a plurality of deep neural network models with different structures is calculated by the simulator, and a deep neural network model with an overall time delay meeting a target time delay condition is determined; comprising: determining a total target speedup ratio according to a target accuracy requirement of the deep neural network model; the total target speedup ratio represents the performance improvement degree of the deep neural network model before and after optimization; The deep neural network model is iteratively optimized by the simulator for multiple rounds, and a sub-target speedup ratio of each iteration round is determined according to the iteration round and the total target speedup ratio; In the current iteration round, the simulator calculates the overall time delay of at least one deep neural network model with different structures based on the time delay data set, and determines a deep neural network model meeting the sub-target speedup ratio of the current round and meeting a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall time delay; The deep neural network model after the current round of optimization is iteratively optimized in the next round until the overall time delay of the deep neural network model meets the total target speedup ratio. 2.The method of Claim 1, wherein, In the current iteration round, the simulator calculates the overall time delay of a plurality of deep neural network models with different structures based on the time delay data set, and determines a deep neural network model meeting the sub-target speedup ratio of the current round and meeting a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall time delay; comprising: determining the number of convolution kernels that need to be reserved for each convolutional layer in the deep neural network model by the simulator, so as to meet the sub-target speedup ratio of the current iteration round; determine the convolution kernel to be reserved based on the influence degree of different convolution kernels in the convolution layer on the accuracy of the deep neural network model, and obtain a candidate deep neural network model corresponding to the convolution layer and meeting the preset accuracy condition; select the deep neural network model with the highest accuracy from the k candidate deep neural network models corresponding to the k trimmable convolution layers to obtain the deep neural network model after the current round of optimization. 3.The method of Claim 1, wherein, In the current iteration round, the simulator calculates the overall latency of the deep neural network model of at least one structure based on the latency data set, and determines the deep neural network model meeting the sub-target acceleration ratio of the current round and the preset accuracy condition as the deep neural network model after the current round of optimization based on the overall latency, including: In the current iteration round, the importance score of each convolution kernel in the deep neural network model after the last round of optimization is calculated; the importance score of the convolution kernel represents the influence degree of the convolution kernel on the accuracy of the deep neural network model; based on the importance score of each convolution kernel in the deep neural network model, and the sub-target acceleration ratio of the current round and the preset accuracy condition, the redundant convolution kernel of the current iteration round is determined globally at one time; the redundant convolution kernel is deleted in the manner of hiding the redundant convolution kernel by a binary mask, and the deep neural network model meeting the overall latency of the sub-target acceleration ratio of the current round and the preset accuracy condition is obtained as the deep neural network model after the current round of optimization, while the convolution kernel hidden by the binary mask still participates in the next iteration round based on its weight. 4.The method of Claim 1, wherein, The optimization method further includes: retain the deep neural network model output by the latest preset number of iterations as a rollback model in a rollback model pool; when the model accuracy of the deep neural network model output by the current iteration round does not meet the preset accuracy condition, select the rollback model with the highest accuracy in the rollback model pool as a rollback target model; based on the rollback target model, re-perform the iteration optimization of the current round. 5.The method for optimizing a deep neural network model of claim 1, wherein, Collect the latency data of the convolution layer with different numbers of convolution kernels on the target platform, and obtain a latency data set including the latency data of each convolution layer with different numbers of convolution kernels, including: collect multiple candidate latency data of the convolution layer with different numbers of convolution kernels on the target platform; select the smallest candidate latency data corresponding to each number of convolution kernels as the latency data of the convolution layer, and obtain the latency data of the convolution layer with different numbers of convolution kernels.

6. An apparatus for optimizing a deep neural network model, characterized by The device includes: an output module configured to extract each convolution layer of the deep neural network model respectively, and output the extracted convolution layers on the target platform respectively; The collection module is configured to, for each convolutional layer, traverse a number of convolution kernels of the convolutional layer, collect time delay data of the convolutional layer with different numbers of convolution kernels on the target platform respectively, and obtain a time delay data set including the time delay data of each convolutional layer with different numbers of convolution kernels; and the collection of the time delay data of the convolutional layer with different numbers of convolution kernels on the target platform comprises: determining a target operator matching the type of the target platform from a plurality of types of operators preconfigured by the simulator; wherein the type of the target platform comprises a CPU, a GPU, an NPU, and an FPGA; the plurality of types of operators preconfigured by the simulator comprise a CPU operator, a GPU operator, an NPU operator, and an FPGA operator; and the time delay data of the convolutional layer with different numbers of convolution kernels on the target platform is collected by using the simulator of the target operator. The calculation module is configured to calculate overall time delays of a plurality of deep neural network models with different structures respectively by using the simulator based on the time delay data set, and determine a deep neural network model with an overall time delay satisfying a target time delay condition; wherein at least one convolutional layer in the deep neural network model with different structures has different numbers of convolution kernels. The training module is configured to retrain the selected deep neural network model to obtain a target deep neural network model special for the target platform. The calculation of the overall time delays of the plurality of deep neural network models with different structures respectively by using the simulator based on the time delay data set, and the determination of the deep neural network model with the overall time delay satisfying the target time delay condition, comprise: determining a total target speedup ratio according to a target accuracy requirement of the deep neural network model; the total target speedup ratio represents a performance improvement degree of the deep neural network model before and after optimization; determining a sub-target speedup ratio of each iteration round according to the iteration round and the total target speedup ratio by performing a plurality of rounds of iteration optimization on the deep neural network model by using the simulator; in the current iteration round, the simulator calculates the overall time delay of the deep neural network model with at least one structure based on the time delay data set, and determines a deep neural network model satisfying the sub-target speedup ratio of the current round and a preset accuracy condition as the deep neural network model after the current round of optimization based on the overall time delay; performing the next round of iteration optimization on the deep neural network model after the current round of optimization until the overall time delay of the deep neural network model satisfies the total target speedup ratio.

7. An electronic device, comprising: The processor, the memory, and the bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the optimization method of the deep neural network model in any one of claims 1 to 5. The computer readable storage medium stores a computer program, the computer program is executed by the processor to perform the steps of the optimization method of the deep neural network model in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, ​

Images