Deep neural network model compression training method, device and apparatus and medium

Abstract

The invention discloses a deep neural network model compression training method, device and apparatus, and a medium. The method comprises the steps of obtaining a target training data set; inputting the target training data set into a pre-constructed first deep neural network model and a pre-constructed second deep neural network model to obtain a first output and a second output; constructing a target loss function according to the first output and the second output; updating a first full-precision weight parameter of the first deep neural network model and a second full-precision weight parameter of the second deep neural network model by using the target loss function; and updating a quantization weight parameter of the second deep neural network model by utilizing the second full-precision weight parameter, and taking the second deep neural network model as a trained compression neural network model when the target loss function meets a preset requirement. Therefore, the size of the model can be reduced, the storage and memory bandwidth requirements are reduced, and the calculation cost is reduced.

Description

technical field [0001] The present application relates to the technical field of machine learning, in particular to a deep neural network model compression training method, device, equipment, and medium. Background technique [0002] Driven by technologies such as large data sets and parallel computing, the neural network based on deep learning has received extensive attention from academia and industry, and has achieved relatively large breakthroughs in many fields, such as image classification, object detection, semantic segmentation, Face detection and face recognition, etc. However, current algorithms based on deep neural network models still face many challenges. First of all, although the algorithm based on the deep neural network model has achieved relatively excellent recognition rates in multiple data sets, the current deep neural network model has a huge amount of parameters, which leads to huge storage and memory bandwidth requirements, resulting in too much resou...

AI Technical Summary

Problems solved by technology

[0002] Driven by technologies such as large data sets and parallel computing, the neural network based on deep learning has received extensive attention from academia and industry, and has achieved relatively large breakthroughs in many fields, such as image classification, object detection, semantic segmentation, Face detection and face recognition, etc. However, the current algorithms based on deep neural network models still face many challenges

First of all, although the algorithm based on the deep neural network model has achieved relatively excellent recognition rates in multiple data sets, the current deep neural network model has a huge amount of parameters, which leads to huge storage and memory bandwidth requirements, resulting in too much resource usage. high

Secondly, the model based on the deep neural network has high computational complexity, which makes it difficult for the algorithm based on the deep neural network model to meet the real-time requirements in terms of inference speed, and is not suitable for devices with high real-time requirements.

The huge amount of parameters and calculations make it difficult to deploy algorithms based on deep neural network models to resource-constrained devices, such as mobile phones, wearable devices, and drones, etc., which greatly limits the use of algorithms based on deep neural network models in various fields. field application

Images