A Deep Neural Network Compression Method Based on Block Item Tensor Decomposition

Abstract

The invention discloses a deep neural network compression method based on block item tensor decomposition. It includes obtaining the deep neural network framework, converting the weight matrix W and the input vector x into high-order tensors W and higher order tensors X , for higher-order tensors W Perform block item tensor decomposition processing, replace the fully connected layer of the deep neural network with the block item tensor layer, and use the back propagation algorithm to train the replaced deep neural network in step D. The present invention adopts the block item tensor decomposition method to construct the block item tensor layer to replace the fully connected layer in the original deep neural network, and utilizes the characteristics of "symmetry" and "exponential expression ability" of the block item tensor layer, not only The parameters of the fully connected layer can be greatly compressed and the classification accuracy of the original network can be maintained.

Description

technical field [0001] The invention belongs to the technical field of deep neural networks, and in particular relates to a deep neural network compression method based on block item tensor decomposition. Background technique [0002] In recent years, deep learning represented by deep neural network has achieved significantly better results than traditional methods in artificial intelligence fields such as image classification, speech recognition, and natural language processing, and has attracted widespread attention from academia, industry, and the government. However, the deep neural network has a complex structure, a large number of layers, and a huge amount of parameters. The trained model often takes up a large storage space and is difficult to port to a small terminal; and the time complexity is also high, even on a high-performance graphics processor. Online training often takes days, which forces us to compress deep neural networks. [0003] Since the weights of de...

AI Technical Summary

Problems solved by technology

However, due to the characteristics of "asymmetry" and "linear expressiveness" of the tensor train decomposition method itself, it is difficult for the compressed deep neural network to obtain higher classification accuracy.

Images