Deep neural network compression method and system and terminal equipment

A technology of deep neural network and compression method, applied in the field of compression method of deep neural network, system and terminal equipment, can solve the problems of low classification accuracy and low computing efficiency, and achieve the effect of ensuring classification accuracy and improving computing efficiency.

Active Publication Date: 2019-06-07
PENG CHENG LAB
View PDF7 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In view of this, the embodiment of the present invention provides a deep neural network compression method, system and terminal equipment to solve the problems of low computational efficiency and low classification accuracy in the current method of pruning and compressing deep neural networks. question

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep neural network compression method and system and terminal equipment
  • Deep neural network compression method and system and terminal equipment
  • Deep neural network compression method and system and terminal equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0043] Such as figure 1 As shown, this embodiment provides a compression method of a deep neural network, which is mainly used in audio and video processing equipment, face recognition equipment, and other computer equipment for classifying, detecting, and segmenting audio, video, and images. The foregoing devices may be general terminal devices, mobile terminal devices, embedded terminal devices, or non-embedded terminal devices, which are not limited here. The compression method of the above-mentioned deep neural network specifically includes:

[0044] Step S101: Input test sample data, obtain the original feature map of the L-th layer of the deep neural network, and determine the redundant filter of the L-th layer according to the original feature map of the L-th layer; wherein, L is not A positive integer less than 1.

[0045] It should be noted that the test sample data is set to test the classification accuracy of the deep neural network after compression and the deep ...

Embodiment 2

[0064] Such as image 3 As shown, in this embodiment, step S101 in Embodiment 1 specifically includes:

[0065] Step S201: Input the test sample data into the deep neural network, and process it through the filter of the Lth layer.

[0066] Step S202: Obtain the output results of each filter.

[0067] Step S203: Obtain the original feature map of the L-th layer after superimposing and transposing the output results of the filters.

[0068] In a specific application, in a specific application, the test sample data is input into the deep neural network, after the data is processed through the filter of the L layer, the output results of each filter are output accordingly, and the output results are superimposed and transposed The original feature map of the L layer can be obtained. Exemplarily, the test sample data is 5000 test sample images, that is, the 5000 test sample images are input into the deep neural network, and after passing through the n filters of the L layer, th...

Embodiment 3

[0076] Such as Figure 4 As shown, in this embodiment, step S102 in Embodiment 1 specifically includes:

[0077] Step S301: Find the corresponding channel of the redundant filter according to the redundant filter.

[0078] In a specific application, since the corresponding channel in the feature map of the redundant filter corresponds to the redundant filter, the corresponding redundant channel can be found through the redundant filter.

[0079] Step S302: Cut out the redundant filter from the filter of the L-th layer.

[0080] Step S303: Pruning the corresponding channel of the redundant filter from the original feature map of the L-th layer to obtain a pruned feature map of the L-th layer.

[0081] In a specific application, the redundant filter is cut from the filter of the L-th layer, and the channel corresponding to the redundant filter is cut out from the original feature map of the L-th layer to complete the pruning process, and the obtained is The filter of the prun...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention is suitable for the technical field of computers, and provides a deep neural network compression method and system, and terminal equipment, and the method comprises the steps: inputtingtest sample data, obtaining an original feature map of an Lth layer of a deep neural network, and determining a redundant filter of the Lth layer according to the original feature map of the Lth layer; pruning the Lth layer according to a redundant filter; obtaining an original feature map of the (L + 1) th layer and a feature map of the L-th layer after pruning; inputting the original feature mapof the (L + 1) th layer and the pruned feature map of the Lth layer into a filter learning model, and automatically learning and outputting a reconstruction filter of the (L + 1) th layer through thefilter learning model; inputting the pruned feature map of the L th layer into a reconstruction filter of the L + 1 layer to obtain a target feature map of the (L + 1) th layer, according to the method, carrying out pruning and reconstruction based on the feature map, the filter is automatically learned and reconstructed in combination with the influence of pruning, the classification accuracy of the compressed deep neural network model is ensured while the structural sparsity of the filter is realized, and the calculation efficiency is improved.

Description

technical field [0001] The invention belongs to the technical field of computers, and in particular relates to a deep neural network compression method, system and terminal equipment. Background technique [0002] Deep neural networks (Convolutional Neural Networks, CNNs) have achieved remarkable success in computer vision tasks such as classification, detection, and segmentation by leveraging large-scale network learning with large amounts of data. However, deep neural networks usually occupy a lot of computing resources and storage space, making their deployment on resource-constrained devices such as mobile and embedded difficult. In order to reduce computing and storage costs, many research works have compressed deep neural network models from the perspective of storage and speed-up. Compression methods include pruning, low-rank decomposition, parameter quantization, transformation / compression convolution kernels, and compact network structure design. [0003] As an eff...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08
Inventor 柳伟仪双燕杨火祥
Owner PENG CHENG LAB
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap