Model acceleration method and apparatus based on knowledge distillation and nonparametric convolution

Abstract

The invention discloses a model acceleration method and a device based on knowledge distillation and nonparametric convolution, wherein, the method comprises the following steps: a cutting step of cutting a nonlinear layer of a convolution neural network and a convolution layer of aggregation redundancy; First distillation step of distilling the original model after cutting the convolution neuralnetwork to obtain an initial convolution neural network; replacing the remaining convolution layers in the first network with non-parametric convolution layers; a second distillation step of maintaining the accuracy of the replaced model by knowledge distillation to obtain a final convolution nerve. This method uses knowledge distillation to obtain lightweight nonparametric convolution, which reduces the model size and improves the runtime speed.

Description

technical field [0001] The invention relates to the field of deep learning technology, in particular to a model acceleration method and device based on knowledge distillation and non-parametric convolution. Background technique [0002] In recent years, convolutional neural networks have achieved breakthrough improvements in a large number of machine learning fields, such as image classification, object detection, semantic segmentation, and speech recognition. Research progress in recent years has shown that the accuracy of convolutional neural networks can be improved by increasing the depth and width of the network. Despite the success of convolutional neural networks, the deployment of networks in real-life applications, especially on mobile devices or embedded On portable devices, they are mainly limited by their large-scale parameters and calculation times. In order to solve this problem, some deep neural network compression algorithms are proposed to learn efficient c...

AI Technical Summary

Problems solved by technology

Research progress in recent years has shown that the accuracy of convolutional neural networks can be improved by increasing the depth and width of the network. Despite the success of convolutional neural networks, the deployment of networks in real-life applications, especially on mobile devices or embedded On portable devices, they are mainly limited by their large-scale parameters and the number of calculations

Images