Deep neural network compression method based on block item tensor decomposition

Abstract

The present invention discloses a deep neural network compression method based on block item tensor decomposition. The method comprises the steps of: obtaining a framework of a deep neural network, respectively converting a weight matrix W and an input vector x to a high-order tensor (u)W(/u) and a high-order tensor (u)X(/u), performing block item tensor decomposition processing of the high-ordertensor (u)W(/u), replacing a full-connection layer of the deep neural network with a block item tensor layer, and employing a back-propagating algorithm to perform training of the deep neural networkreplaced in the step D. The deep neural network compression method based on the block item tensor decomposition employs a block item tensor decomposition method to construct the block item tensor layer to replace the full- connection layer of the original deep neural network, the features of symmetry and index expression capability of the block item tensor layer are employed to greatly compress the parameter amount of the full-connection layer and maintain the classification precision of the original network.

Description

technical field [0001] The invention belongs to the technical field of deep neural networks, and in particular relates to a deep neural network compression method based on block item tensor decomposition. Background technique [0002] In recent years, deep learning represented by deep neural network has achieved significantly better results than traditional methods in artificial intelligence fields such as image classification, speech recognition, and natural language processing, and has attracted widespread attention from academia, industry, and the government. However, the deep neural network has a complex structure, a large number of layers, and a huge amount of parameters. The trained model often takes up a large storage space and is difficult to port to a small terminal; and the time complexity is also high, even on a high-performance graphics processor. Online training often takes days, which forces us to compress deep neural networks. [0003] Since the weights of de...

AI Technical Summary

Problems solved by technology

However, due to the characteristics of "asymmetry" and "linear expressiveness" of the tensor train decomposition method itself, it is difficult for the compressed deep neural network to obtain higher classification accuracy.

Images