Sensitivity analysis and reinforcement learning neural network pruning method, system and device

A sensitivity analysis and reinforcement learning technology, applied in the field of neural network pruning, it can solve the problems of low network accuracy and inability to contain training data, and achieve the effect of improving the compression rate

Pending Publication Date: 2021-01-05
深兰人工智能(深圳)有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The present invention proposes a neural network pruning method based on sensitivity analysis and reinforcement learning in order to sol

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sensitivity analysis and reinforcement learning neural network pruning method, system and device
  • Sensitivity analysis and reinforcement learning neural network pruning method, system and device
  • Sensitivity analysis and reinforcement learning neural network pruning method, system and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0041] Using sensitivity analysis (sensitivity analysis) to analyze the neural network, the initial data in the data buffer of reinforcement learning is randomly carried out within a certain range.

[0042] Through the sensitivity analysis, it can be determined which weights are highly sensitive and cannot be pruned, and those weights whose sensitivity is too low can be pruned.

[0043] A neural network pruning method for sensitivity analysis and reinforcement learning proposed by the present invention, such as figure 1 shown, including:

[0044] S100. Select low-sensitivity weights for pruning, set these selected weights as W(w0, w1, w2,...wn), and then set the sparsity threshold T(t0, t1, t2 of each weight ...tn). The selection of these thresholds must ensure that the accuracy of the network drop remains within 20% after the clipped weights are clipped with the current sparsity.

[0045] S200. Determine the weights that need to be randomly pruned according to the W obtain...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sensitivity analysis and reinforcement learning neural network pruning method, system and device, and the method comprises the steps: setting a sparsity threshold, and selecting a low-sensitivity weight for pruning; obtaining a cutting method and precision, and determining a weight needing random pruning according to the sensitivity weight; carrying out random clipping oneach selected weight, and putting a pruning method and precision of multiple times of random clipping into a buffer area; a training reinforcement learning step: training a reinforcement learning agent by using the data in the buffer area, and putting a cutting method and precision generated after training into the buffer area; and repeating until the network precision reaches a preset value. According to the method, the low-sensitivity weight is selected for pruning, and the sparsity threshold value of each weight is set to ensure that the network descending precision is kept within the preset range after the cut weight is cut by adopting the current sparsity process. Under the condition that the network precision is ensured, the compression ratio of the neural network is improved to themaximum extent.

Description

technical field [0001] This application relates to the field of deep learning compression technology, in particular, to a neural network pruning method based on sensitivity analysis and reinforcement learning. Background technique [0002] Pruning (prune) is a compression technique of convolutional neural network (CNN), which is mainly used to reduce the calculation amount of convolutional neural network (CNN). The pruning algorithm usually achieves the purpose of reducing the calculation amount of the entire neural network by cutting out unimportant tensors in the weight of the neural network. [0003] Which tensors in the neural network weights are not important are determined by their sparsity. Sparsity is used to measure the number of 0s in the tensor and the size of the tensor. Therefore, cutting out the tensor with higher sparsity in the weight can achieve the purpose of compressing the convolutional neural network (CNN). [0004] The principle of convolutional neur...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06N3/04G06N3/063
CPCG06N3/04G06N3/063G06N3/082
Inventor 陈海波关翔
Owner 深兰人工智能(深圳)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap