Neural network filter pruning method based on batch feature heat map

A neural network and filter technology, applied in the fields of neural computing optimization, artificial intelligence and computer vision, can solve the problems of insufficient model accuracy and size, and achieve the effect of compressing model size, improving inference speed, and efficient pruning.

Active Publication Date: 2021-03-26
INST OF OPTICS & ELECTRONICS - CHINESE ACAD OF SCI
View PDF10 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since this method prunes the network with the filter parameter size as an absolute reference, there are still deficiencies in the accuracy and size of the pruned model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network filter pruning method based on batch feature heat map
  • Neural network filter pruning method based on batch feature heat map
  • Neural network filter pruning method based on batch feature heat map

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] In order to more clearly illustrate the purpose, technical solutions and advantages of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0046] Taking the public data set CIFAR-10 and the classic network model VGG16 pre-trained on ImageNet as examples, the specific implementation of a neural network filter pruning method based on batch feature heatmaps of the present invention will be further described in detail in conjunction with the accompanying drawings ,in figure 1 It is a flow chart of the overall structure for realizing filter pruning through the present invention.

[0047] Step 1: Load the pre-trained convolutional neural network model VGG16 and its related configuration files, and fine-tune the pre-trained model for dozens of rounds based on the CIFAR-10 dataset to obtain the fine-tuned model.

[0048] Step 2: Randomly extract 128 images from the CIFA...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a neural network filter pruning method based on a batch feature heat map. The method is mainly used for reducing the model memory space and improving the model reasoning speed.The method comprises the following steps: loading and finely adjusting a pre-training model on a given data set; generating a batch feature heat map of each layer of the model; obtaining Mask of eachfilter based on the gray threshold to score the filters; performing random non-repetitive extraction on the given data set to update the score of the filter; realizing pruning of each layer of filterby taking the score of the filter as a measurement criterion; and re-training the pruned model to recover the precision, and the like. According to the invention, the problems of large storage capacity and low reasoning speed of the neural network model are solved, so that the pruned neural network model can be applied to a resource-limited scene under the condition of extremely low precision reduction.

Description

technical field [0001] The invention relates to the fields of artificial intelligence, computer vision, and neural computing optimization, in particular to a neural network filter pruning method based on a batch feature heat map, which prunes the neural network filter to reduce model size, reduce calculation consumption and speed up Model compression methods for network inference speed. Background technique [0002] In recent years, deep neural networks have made great progress in many application scenarios with their powerful representation capabilities, such as great progress in intelligent research on images and videos, speech and text. But behind the success of the deep neural network is the powerful and complex neural network model, which consumes a lot of storage space and computing resources. For example, the AlexNet model for image recognition proposed in 2012 contains 600,000 parameters and about 240MB of memory, and it needs to use multiple GPUs to train about 1.2...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045
Inventor 罗辉张建林徐智勇李红川
Owner INST OF OPTICS & ELECTRONICS - CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap