Compression method for convolutional neural network based on global error reconstruction

Abstract

A compression method for a convolutional neural network based on global error reconstruction relates to compression of a deep neural network. Aiming at the defect that a high-precision classificationeffect cannot be obtained in a conventional intra-laminar compression technology based on low-rank decomposition, considering various inter-laminar non-linear relations, replacing single-layer optimization with inter-laminar joint optimization of parameters, a global error minimization optimization scheme is constructed and the compression method for the convolutional neural network constructed based on the global error is provided. The method comprises the following steps of (1) initially compressing the size of a mold by use of intra-laminar linear response low-rank decomposition method without considering a non-linear activation function; (2) establishing a non-linear intra-laminar compression optimization in consideration of the influence of non-linear activation on a single layer by use of low-rank decomposition of a network intra-laminar matrix to improve the compression precision of a non-linear intra-laminar matrix; and (3) constructing the global error reconstruction to improve the global discrimination capability of a compression model, wherein an intra-laminar compression error increases layer by layer.

Description

technical field [0001] The invention relates to the compression of deep neural networks, in particular to a compression method of convolutional neural networks based on global error reconstruction. Background technique [0002] In recent years, with the rapid development of hardware GPU and the advent of the era of big data, deep learning has developed rapidly and has swept all fields of artificial intelligence, including speech recognition, image recognition, video tracking, natural speech processing, etc. , Video field. Deep learning technology breaks through traditional technical methods and greatly improves the recognition performance in various fields, especially the powerful self-feature representation ability of convolutional neural networks (CNNs), which makes it widely used in image recognition[1-4] , target detection [5-7], image retrieval [8] and other fields. If the powerful recognition performance of the convolutional neural network can be transplanted into mo...

AI Technical Summary

Problems solved by technology

[0024] The purpose of the present invention is to address the shortcomings of the traditional low-rank decomposition-based intra-layer compression technology that cannot obtain high-precision classification effects, consider various non-linear relationships between layers, and use joint optimization between parameter layers to replace single-layer optimization. Global error minimization optimization scheme, providing a compression method for convolutional neural networks based on global error reconstruction

Images